feat: add auth0-checkmate + auth0-healthcheck-all-plans self-audit skills#127
Conversation
… skill Adds the auth0-healthcheck-all-plans skill: a plan-aware Auth0 tenant health check (Free through Enterprise) that fuses CheckMate-style security/config hygiene with use-case capability fit, emits two scores, recommends a specific self-service plan with cost or an "Enterprise — contact sales" path, and can optionally apply approved fixes via the Auth0 CLI with per-command confirmation. - SKILL.md plus references/ (decision logic, pricing source of truth, scoring model, report templates) - scripts/render_pdf.sh (HTML to PDF report renderer) - documented in plugins/auth0/README.md Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Caution Review failedAn error occurred during the review process. Please try again later. Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds two Auth0 skills and expands their offline references, scoring, plan-routing rules, validation constraints, report templates, and PDF rendering scripts. ChangesAuth0 Checkmate and Healthcheck Skills
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 15
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.md`:
- Around line 31-45: Fix the markdownlint violations (MD031/MD040) throughout
the auth0-cli-readme.md file by ensuring all fenced code blocks are properly
formatted. At lines 31-45, 58-72, 119-123, and 206-209, add blank lines before
and after each fenced code block if missing (to satisfy MD031), and ensure every
code fence has an appropriate language tag (such as bash, shell, or other
relevant language) specified after the opening triple backticks (to satisfy
MD040). For any "bare" fences without language identifiers, add the correct
language tag to indicate the code type being demonstrated.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/data-integrity-rules.md`:
- Around line 87-100: The data integrity rules contain two logical errors that
can cause incorrect plan recommendations. First, the comparison `current_plan <
"Professional"` uses lexical string comparison instead of explicit Auth0 plan
tier ranking, which will produce incorrect results. Replace this with an
explicit plan ranking check that properly compares plans by their tier level
(e.g., Essentials < Professional < Enterprise). Second, the predicate `use_case
== "AI-Native" OR "AI-Differentiated" AND integrations.length > 0` lacks
parentheses and relies on operator precedence, creating ambiguity. Wrap the
use_case equality checks in parentheses to make the logical intent clear:
`(use_case == "AI-Native" OR use_case == "AI-Differentiated") AND
integrations.length > 0` to ensure B2B and AI tenants are correctly routed.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/enterprise-need-detection.md`:
- Around line 14-30: Remove the line containing "M2M Access for Organizations
(B2C)," from the enterprise features list in the enterprise-need-detection.md
file. This feature is already handled as a B2B Professional unlock in
plan-matching-logic.md, so including it in the Class A enterprise-only trigger
list causes self-service tenants to be incorrectly short-circuited to the
Enterprise path when they should remain self-service. Delete this specific entry
from the conditional feature list to align with the existing plan-matching
logic.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/feature-unlock-matrix.md`:
- Around line 255-300: The feature availability table contains inaccuracies
compared to the actual pricing.md source. Update the Log Streaming row in the
Security & Compliance Features table to show 2 streams (not 1 stream) for the
B2C Professional, B2B Professional, and Enterprise columns. Additionally, update
the M2M Tokens row in the M2M & Developer Features table to show "Add-on" (not
"No") for the B2B Essentials column to properly reflect that M2M tokens can be
purchased as an add-on at that plan level. This ensures the table accurately
represents the included-vs-add-on distinction from pricing.md and prevents
understating real entitlements.
- Around line 3-4: The Custom Domain rows in the "Free → ..." unlock sections
contradict the header note which states that Free tier already includes 1 Custom
Domain. Remove Custom Domain as an upgrade item from all "Free → ..." sections
(lines 6-17, 25-41, 44-56, 64-83, 127-140, 259-265) and instead mark those rows
as "already included" to align with the pricing.md source of truth. Similarly,
for Enterprise plan rows showing Custom Domain, change them to reflect
"included" rather than inventing an "unlimited" upgrade that pricing.md does not
specify.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/mau-forecast-calculator.md`:
- Around line 58-77: The Interactive Fallback Logic section currently asks for
growth rate before requesting current MAU, but growth rate alone cannot produce
a valid forecast without a starting point. Restructure the flow in the
"Interactive Fallback Logic" section to first collect and verify the current MAU
value from the user (either from tenant telemetry or by prompting the user
directly) before asking for the monthly growth rate. Only after both current MAU
and growth rate are obtained should the forecast calculation proceed. This
ensures the prerequisite data (current MAU) is available before attempting to
apply growth projections.
- Around line 35-45: The forecast values shown in the example do not match the
compound growth formula 500 × 1.15^N. Recalculate and update all forecast values
throughout the document using the correct formula: Month 6 should be
approximately 1,157 (currently 1,005), Month 12 should be approximately 2,675
(currently 2,456), Month 24 should be approximately 14,313 (currently 12,000),
and Month 28 should be approximately 25,033 (currently 25,000). Apply these
corrected values consistently wherever the forecast timeline appears in the
mau-forecast-calculator.md file.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/plan-matching-logic.md`:
- Around line 64-95: The plan-matching-logic.md file contains hardcoded MAU
ceiling values (40k and 50k) in the comparison logic for both B2C Essentials and
B2C Professional tiers (at lines 64-95, 103-146, and 194-201) that do not align
with track-specific ceiling limits defined elsewhere in the skill, causing
incorrect routing for B2B customers. Replace all hardcoded MAU thresholds with
references to the appropriate track-specific ceiling variables or constants.
Ensure that the readiness scoring logic and upgrade recommendation conditions at
each affected location use the correct per-track limits instead of the generic
hardcoded values, and verify that the REASON fields explaining MAU ceiling
constraints reference the actual track-specific ceilings rather than the
outdated hardcoded "50k" value.
- Around line 34-46: The A4AA fit-score gate is not being applied consistently
across all plan recommendations in the decision logic. The FREE and B2B
Essentials branches currently recommend the A4AA add-on without checking the
documented a4aa_fit_score ≥ 0.4 threshold, while B2B Professional and Enterprise
branches use the incorrect boundary operator > 0.4 instead of ≥ 0.4. Apply the
a4aa_fit_score ≥ 0.4 check as a gate condition before recommending A4AA add-ons
in all plan branches (FREE, B2B Essentials, B2B Professional, and Enterprise) to
ensure consistent behavior across all recommendation paths and prevent
over-selling or suppressing the add-on against the skill contract.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/remediation-command-map.md`:
- Around line 3-38: The document currently specifies a universal fallback to
`auth0 api patch` which does not work for PUT-only resources (MFA factor
toggles, MFA policies, Prompts customization) or POST-only operations. Update
the initial instruction paragraph that mentions "fall back to `auth0 api patch
<path>`" to clarify that the correct HTTP method (PATCH, PUT, or POST) should be
used based on the endpoint requirements. Similarly, update the final sentence
stating "The universal fallback is `auth0 api patch <path> --data '<json>'" to
specify that agents should use the appropriate HTTP method from the
command-shape mapping table for each resource category, or explicitly add
fallback command rows for POST and PUT resources in the table where they differ
from PATCH.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.html`:
- Around line 2-7: The HTML comment header in the report template references the
wrong path for the PDF rendering helper script. Update the path reference in the
comment from `bin/render_pdf.sh` to `scripts/render_pdf.sh` to correctly point
to the actual location of the render_pdf.sh script added in this PR.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.md`:
- Around line 8-9: The instructional comments in the report templates contain
example placeholder tokens (like `{{customer_name}}` and `{{reviewer_org}}`)
that will survive the substitution process and cause the generated output to
fail Fusion linting. In
`plugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.md`
at lines 8-9, rewrite the inline comment on line 9 (the prepared_by_line
instruction) to explain the conditional logic without using placeholder syntax.
In
`plugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.html`
at lines 98-99, rewrite the subtitle note and any other instructional comments
to remove all `{{...}}` examples and replace them with plain text descriptions
of what the placeholders represent.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/scoring-model.md`:
- Around line 14-26: The scoring model contains a fallback that estimates the
total number of checks as approximately 50 from CheckMate's documented total,
which can become inaccurate as CheckMate evolves. Remove the intermediate
fallback that uses the approximate "~50-check total" (the section describing
`weighted_total = weighted_failures + (total_checks − `#failures`)` with the
caveat about CheckMate's documented count). Instead, strengthen the guidance to
explicitly state that if real passing-count data or total-check counts cannot be
extracted from the CheckMate `summary[]` output by the parser, the hygiene score
must be marked as "Not scored" rather than falling back to estimation. This
ensures that the model only produces scores when it has actual verified data
from the upstream report, preventing misleading estimates that drift with
CheckMate changes.
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/use-case-detection-logic.md`:
- Around line 93-95: The CIBA acronym expansion in the markdown file is
incorrect. Replace the current expansion "Confirmation by External Analyzer"
with the correct expansion "Client Initiated Backchannel Authentication" in the
line that begins with "CIBA" to ensure accurate terminology is used in the
report.
In `@plugins/auth0/skills/auth0-healthcheck-all-plans/scripts/render_pdf.sh`:
- Around line 83-102: The render_pdf script checks if OUTPUT_ABS exists to
determine success, but this doesn't account for stale files from previous failed
runs. A failed Chrome execution will leave the old file in place, incorrectly
indicating success. Either delete OUTPUT_ABS before attempting the Chrome render
operations, or redirect Chrome to write to a temporary file first and only move
it to OUTPUT_ABS if the render succeeds (after both Chrome attempts). This
ensures that a successful exit only occurs when Chrome actually produces new
output, not when a leftover file happens to exist.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a1483b0e-2e6d-489e-886c-033cf3b09ebc
📒 Files selected for processing (19)
plugins/auth0/README.mdplugins/auth0/skills/auth0-healthcheck-all-plans/SKILL.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/checkmate-readme.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/data-integrity-rules.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/enterprise-need-detection.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/feature-recommendations.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/feature-unlock-matrix.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/mau-forecast-calculator.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/plan-matching-logic.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/pricing.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/production-readiness-checklist.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/remediation-command-map.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.htmlplugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/scoring-model.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/talk-to-sales-block.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/use-case-detection-logic.mdplugins/auth0/skills/auth0-healthcheck-all-plans/scripts/render_pdf.sh
…tes it Adds auth0-checkmate: an Auth0 CheckMate tenant audit that bootstraps the Auth0 CLI plus a dedicated M2M app, gathers lightweight public company context inline (folded in from the former standalone enrichment helper, sanitized to drop sales-qualification fit scores, plan-fit verdicts, and firmographics), and produces a personalized Platform Development Review (markdown + styled PDF) with per-command-confirmed remediation. Updates auth0-healthcheck-all-plans to orchestrate only auth0-checkmate as its single sister skill (CheckMate now supplies both the tenant scan and the company context). The health check keeps its use-case-based plan recommendation (Phase 2 classify + Phase 4 plan matching) unchanged. - auth0-checkmate: SKILL.md + references (pricing matrix, report templates, CheckMate/CLI readmes) + scripts/render_pdf.sh - documented in plugins/auth0/README.md Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
auth0-self-serve-monetization
changed the title
Resolves all 15 inline review comments on PR auth0#127: - plan-matching-logic: apply the a4aa_fit_score >= 0.4 gate to every A4AA branch (and use >= not >); replace hardcoded 40k/50k MAU ceilings with track-specific limits from pricing.md - feature-unlock-matrix: stop presenting the Free-included Custom Domain as a paid unlock; correct Log Streaming (2 streams on Professional) and M2M-on-Essentials (Add-on); Enterprise custom domains "Included" not "Unlimited" - enterprise-need-detection: drop "M2M Access for Organizations" (a Professional unlock) from the Enterprise-only trigger list - data-integrity-rules: parenthesize the AI predicate; compare plans by explicit tier rank, not lexical string comparison - scoring-model: derive the Hygiene denominator from the real parsed finding count, never an estimate; note the flat-array CheckMate report shape - mau-forecast-calculator: recompute the sample timeline; collect current MAU before asking for growth rate - remediation-command-map: fallback uses the correct HTTP method (PATCH/PUT/POST) per resource instead of a universal PATCH - report-template (md + html): strip {{...}} from instructional comments so they cannot survive substitution and trip the fusion lint; fix the render_pdf path - render_pdf.sh: render to a temp file and promote only on a non-empty result (don't treat a stale PDF as a successful render) - use-case-detection-logic: correct the CIBA expansion - auth0-cli-readme: markdownlint fence fixes (MD031 blank lines, MD040 language tags) Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review The latest commit (06b235d) addresses all 15 inline review comments from the previous pass — see the commit message for the per-file mapping. |
|
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/auth0/skills/auth0-checkmate/SKILL.md (1)
1-574: 🧹 Nitpick | 🔵 TrivialSKILL.md exceeds best-practice length; splitting phases into references/ is optional but could improve agent portability.
The file spans 573 lines against the 500-line aspiration in coding guidelines. However, skillsaw validation does not enforce a line-count limit — only the Agent Skills directory structure (SKILL.md in root, supporting content in subdirectories). This skill already complies: the
references/directory exists and houses template/config files, whilescripts/contains build logic.Phases 1–7 (lines 124–430) contain detailed procedural runbooks that could migrate to reference files (e.g.,
references/cli-bootstrap-phases.md,references/remediation-phases.md) without loss of clarity. Tradeoff: the current integrated structure provides clear phase boundaries that agents find easy to follow; splitting would require careful cross-referencing to maintain coherence. If agent field feedback indicates the runbook is hard to navigate at this length, refactoring becomes worthwhile; otherwise, the current structure is valid and within spec.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/auth0/skills/auth0-checkmate/SKILL.md` around lines 1 - 574, The SKILL.md file is lengthy (573 lines) which may make it harder for agents to navigate and maintain portability across different platforms. While the file structure is currently valid and complies with skillsaw specs, you can improve agent portability by extracting the detailed procedural runbooks from Phases 1 through 7 (lines 124–430) into separate reference files—for example, creating `references/cli-bootstrap-phases.md` for Phases 1–2, and `references/remediation-phases.md` for Phases 6–7—and then linking to those files from the main SKILL.md with brief anchors and cross-references. This keeps the primary skill document concise and scannable while preserving all detail in organized, reusable reference modules that agents can load independently.Source: Coding guidelines
♻️ Duplicate comments (1)
plugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.md (1)
128-130:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix remaining markdownlint MD040 violations (bare code fences).
Lines 128–130 and 215–218 contain bare code fences without language tags. While these display agent-skill commands (not executable code), markdownlint still requires a language identifier. Add a language tag like
textorbashto each fence (e.g.,```text).The prior review addressed most MD031/MD040 issues, but these two bare fences remain. This is a quick fix to clear linting.
Also applies to: 215-218
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.md` around lines 128 - 130, The markdown file contains bare code fences without language identifiers at lines 128–130 and 215–218 that violate markdownlint MD040 rules. Add a language tag to each bare code fence in the auth0-cli-readme.md file by changing ``` to ```text (or ```bash if appropriate) at both locations to comply with linting requirements and clear the remaining violations.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/auth0/skills/auth0-checkmate/references/auth0-cli-readme.md`:
- Around line 31-119: The markdown file contains markdownlint violations that
need to be fixed: First, add a blank line before each opening code fence (```)
when the previous line is not blank to satisfy MD031 requirement. Second, add
explicit language identifiers to all code fences that are missing them (such as
bash, powershell, text, etc.) to satisfy MD040 requirement. Specifically, ensure
all code blocks in the installation sections and authentication sections have
proper blank line spacing and language tags specified after the opening fence
markers.
---
Outside diff comments:
In `@plugins/auth0/skills/auth0-checkmate/SKILL.md`:
- Around line 1-574: The SKILL.md file is lengthy (573 lines) which may make it
harder for agents to navigate and maintain portability across different
platforms. While the file structure is currently valid and complies with
skillsaw specs, you can improve agent portability by extracting the detailed
procedural runbooks from Phases 1 through 7 (lines 124–430) into separate
reference files—for example, creating `references/cli-bootstrap-phases.md` for
Phases 1–2, and `references/remediation-phases.md` for Phases 6–7—and then
linking to those files from the main SKILL.md with brief anchors and
cross-references. This keeps the primary skill document concise and scannable
while preserving all detail in organized, reusable reference modules that agents
can load independently.
---
Duplicate comments:
In
`@plugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.md`:
- Around line 128-130: The markdown file contains bare code fences without
language identifiers at lines 128–130 and 215–218 that violate markdownlint
MD040 rules. Add a language tag to each bare code fence in the
auth0-cli-readme.md file by changing ``` to ```text (or ```bash if appropriate)
at both locations to comply with linting requirements and clear the remaining
violations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: f15df398-775a-4ee0-aae5-667693e6c414
📒 Files selected for processing (21)
plugins/auth0/README.mdplugins/auth0/skills/auth0-checkmate/SKILL.mdplugins/auth0/skills/auth0-checkmate/references/auth0-cli-readme.mdplugins/auth0/skills/auth0-checkmate/references/auth0-pricing.mdplugins/auth0/skills/auth0-checkmate/references/checkmate-readme.mdplugins/auth0/skills/auth0-checkmate/references/report-template.htmlplugins/auth0/skills/auth0-checkmate/references/report-template.mdplugins/auth0/skills/auth0-checkmate/scripts/render_pdf.shplugins/auth0/skills/auth0-healthcheck-all-plans/SKILL.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/auth0-cli-readme.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/data-integrity-rules.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/enterprise-need-detection.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/feature-unlock-matrix.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/mau-forecast-calculator.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/plan-matching-logic.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/remediation-command-map.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.htmlplugins/auth0/skills/auth0-healthcheck-all-plans/references/report-template.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/scoring-model.mdplugins/auth0/skills/auth0-healthcheck-all-plans/references/use-case-detection-logic.mdplugins/auth0/skills/auth0-healthcheck-all-plans/scripts/render_pdf.sh
💤 Files with no reviewable changes (1)
- plugins/auth0/skills/auth0-healthcheck-all-plans/references/enterprise-need-detection.md
…s with the sanitized skill CodeRabbit's re-review of PR auth0#127 flagged the vendored CLI README in the checkmate skill; these were the same latent issues already fixed on the healthcheck side (06b235d) in checkmate's sibling files: - scripts/render_pdf.sh: render to a temp file, promote only on a non-empty result (don't treat a stale PDF as a successful render) - references/auth0-cli-readme.md: markdownlint fences (MD031 blank lines, MD040 language tags) - references/report-template.{md,html}: bin/ -> scripts/ render path; strip {{...}} from instructional comments so they can't survive substitution and trip the fusion lint Also aligns the report templates with the skill's already-sanitized recommendation logic — the templates still rendered the removed sales fields: - header "Auth0 Fit / {{auth0_plan_recommendation}}" -> "Recommended Plan / {{recommended_plan}}"; Section 4 banner likewise (md + html) - source-of-truth notes now point to the Phase 3 company context (sanitized: no fit scores, no firmographics), with recommended_plan / a4aa_yes_no derived from use case + findings + auth0-pricing.md; corrected the flat-array report-shape note Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review Commit c1bc2c8 resolves the markdownlint finding in |
|
✅ Action performedFull review finished. |
| | [auth0-net-ios](skills/auth0-net-ios) | Integrates `Auth0.OidcClient.iOS` into .NET iOS (Xamarin) applications. Covers Web Auth login/logout. | [SKILL.md](skills/auth0-net-ios/SKILL.md) | | ||
| | [auth0-winforms](skills/auth0-winforms) | Integrates `Auth0.OidcClient.WinForms` into .NET WinForms applications. Covers Web Auth login/logout. | [SKILL.md](skills/auth0-winforms/SKILL.md) | | ||
| | [auth0-wpf](skills/auth0-wpf) | Integrates `Auth0.OidcClient.WPF` into .NET WPF applications. Covers Web Auth login/logout. | [SKILL.md](skills/auth0-wpf/SKILL.md) | | ||
| | [auth0-checkmate](skills/auth0-checkmate) | Runs an Auth0 CheckMate tenant audit and produces a personalized configuration / security report (Auth0 Platform Development Review — markdown + styled PDF). Bootstraps the Auth0 CLI and a dedicated CheckMate M2M app, gathers lightweight public company context so findings read in the customer's own terms, triages fixes into immediate vs after-upgrading, and applies approved fixes via the Auth0 CLI with per-command confirmation. | [SKILL.md](skills/auth0-checkmate/SKILL.md) | |
There was a problem hiding this comment.
This description is quite long compared to other entries in the table — worth trimming to a single short sentence so it stays consistent.
| | [auth0-winforms](skills/auth0-winforms) | Integrates `Auth0.OidcClient.WinForms` into .NET WinForms applications. Covers Web Auth login/logout. | [SKILL.md](skills/auth0-winforms/SKILL.md) | | ||
| | [auth0-wpf](skills/auth0-wpf) | Integrates `Auth0.OidcClient.WPF` into .NET WPF applications. Covers Web Auth login/logout. | [SKILL.md](skills/auth0-wpf/SKILL.md) | | ||
| | [auth0-checkmate](skills/auth0-checkmate) | Runs an Auth0 CheckMate tenant audit and produces a personalized configuration / security report (Auth0 Platform Development Review — markdown + styled PDF). Bootstraps the Auth0 CLI and a dedicated CheckMate M2M app, gathers lightweight public company context so findings read in the customer's own terms, triages fixes into immediate vs after-upgrading, and applies approved fixes via the Auth0 CLI with per-command confirmation. | [SKILL.md](skills/auth0-checkmate/SKILL.md) | | ||
| | [auth0-healthcheck-all-plans](skills/auth0-healthcheck-all-plans) | Runs a plan-aware health check of an Auth0 tenant across all plans (Free to Enterprise), producing two scores — Security & Config Hygiene and Capability Fit — plus a use-case-based plan recommendation (a self-service plan with cost, or an Enterprise contact-sales path). Orchestrates the auth0-checkmate skill and can apply approved fixes via the Auth0 CLI with per-command confirmation. | [SKILL.md](skills/auth0-healthcheck-all-plans/SKILL.md) | |
There was a problem hiding this comment.
Same here — description is much longer than others in the table. A single short sentence would be cleaner.
|
|
||
| if command -v wkhtmltopdf >/dev/null 2>&1; then | ||
| echo "render_pdf: using wkhtmltopdf" >&2 | ||
| wkhtmltopdf --enable-local-file-access "$INPUT_ABS" "$OUTPUT_ABS" |
There was a problem hiding this comment.
If wkhtmltopdf fails here the script still exits 0 — the caller won't know the PDF was never written. Worth propagating the exit code.
|
|
||
| if command -v weasyprint >/dev/null 2>&1; then | ||
| echo "render_pdf: using weasyprint" >&2 | ||
| weasyprint "$INPUT_ABS" "$OUTPUT_ABS" |
There was a problem hiding this comment.
Same as above for weasyprint — a failure here silently exits 0.
|
|
||
| if command -v wkhtmltopdf >/dev/null 2>&1; then | ||
| echo "render_pdf: using wkhtmltopdf" >&2 | ||
| wkhtmltopdf --enable-local-file-access "$INPUT_ABS" "$OUTPUT_ABS" |
There was a problem hiding this comment.
If wkhtmltopdf fails here the script still exits 0 — the caller won't know the PDF was never written. Worth propagating the exit code.
|
|
||
| if command -v weasyprint >/dev/null 2>&1; then | ||
| echo "render_pdf: using weasyprint" >&2 | ||
| weasyprint "$INPUT_ABS" "$OUTPUT_ABS" |
There was a problem hiding this comment.
Same as above for weasyprint — a failure here silently exits 0.
There was a problem hiding this comment.
The two skills write state to different directories — auth0-checkmate uses ~/.auth0-checkmate/state/ but auth0-healthcheck-all-plans uses ~/.claude/skills/auth0-healthcheck-all-plans/state/. This means the healthcheck can't read the enrichment cache that checkmate writes, so it'll re-fetch company context on every run. Both should use the same path.
Please take a look: the CI checks are failing, and some commits are unsigned.
| @@ -0,0 +1,573 @@ | |||
| --- | |||
There was a problem hiding this comment.
No tests/ directory — skills like auth0-flutter-native include evals.json and graders.json to validate end-to-end behavior. Worth adding before merge, especially for a skill that touches tenant config.
| @@ -0,0 +1,152 @@ | |||
| --- | |||
There was a problem hiding this comment.
No tests/ directory — same as the sister skill. Some eval coverage for the scoring and plan-recommendation logic would go a long way here.
Resolves all 15 inline review comments on PR auth0#127: - plan-matching-logic: apply the a4aa_fit_score >= 0.4 gate to every A4AA branch (and use >= not >); replace hardcoded 40k/50k MAU ceilings with track-specific limits from pricing.md - feature-unlock-matrix: stop presenting the Free-included Custom Domain as a paid unlock; correct Log Streaming (2 streams on Professional) and M2M-on-Essentials (Add-on); Enterprise custom domains "Included" not "Unlimited" - enterprise-need-detection: drop "M2M Access for Organizations" (a Professional unlock) from the Enterprise-only trigger list - data-integrity-rules: parenthesize the AI predicate; compare plans by explicit tier rank, not lexical string comparison - scoring-model: derive the Hygiene denominator from the real parsed finding count, never an estimate; note the flat-array CheckMate report shape - mau-forecast-calculator: recompute the sample timeline; collect current MAU before asking for growth rate - remediation-command-map: fallback uses the correct HTTP method (PATCH/PUT/POST) per resource instead of a universal PATCH - report-template (md + html): strip {{...}} from instructional comments so they cannot survive substitution and trip the fusion lint; fix the render_pdf path - render_pdf.sh: render to a temp file and promote only on a non-empty result (don't treat a stale PDF as a successful render) - use-case-detection-logic: correct the CIBA expansion - auth0-cli-readme: markdownlint fence fixes (MD031 blank lines, MD040 language tags) Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s with the sanitized skill CodeRabbit's re-review of PR auth0#127 flagged the vendored CLI README in the checkmate skill; these were the same latent issues already fixed on the healthcheck side (06b235d) in checkmate's sibling files: - scripts/render_pdf.sh: render to a temp file, promote only on a non-empty result (don't treat a stale PDF as a successful render) - references/auth0-cli-readme.md: markdownlint fences (MD031 blank lines, MD040 language tags) - references/report-template.{md,html}: bin/ -> scripts/ render path; strip {{...}} from instructional comments so they can't survive substitution and trip the fusion lint Also aligns the report templates with the skill's already-sanitized recommendation logic — the templates still rendered the removed sales fields: - header "Auth0 Fit / {{auth0_plan_recommendation}}" -> "Recommended Plan / {{recommended_plan}}"; Section 4 banner likewise (md + html) - source-of-truth notes now point to the Phase 3 company context (sanitized: no fit scores, no firmographics), with recommended_plan / a4aa_yes_no derived from use case + findings + auth0-pricing.md; corrected the flat-array report-shape note Passes `uvx skillsaw --strict` (0 errors, 0 warnings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sanchitmehtagit
force-pushed
the
add-auth0-healthcheck-all-plans-skill
branch
2 times, most recently
from
84dc628 to
c1bc2c8
Compare
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/auth0/skills/auth0-checkmate/references/report-template.md (1)
115-118: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winLiteral
{{...}}ellipsis can leak into the rendered brief.The list body uses
{{...}}as a "continue the list" hint, but it matches the same{{token}}shape the substitution step processes and has no source mapping, so a naive find/replace leaves a stray{{...}}in the output. This is the same leakage class the PR set out to remove from instructional comments. Move the hint into a non-{{...}}comment (as the HTML template does) or drop the literal token.📝 Proposed fix
1. {{immediate_action_1}} 2. {{immediate_action_2}} 3. {{immediate_action_3}} -{{...}} +<!-- continue numbering for each remaining immediate action -->🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/auth0/skills/auth0-checkmate/references/report-template.md` around lines 115 - 118, The list in report-template.md still contains a literal {{...}} placeholder that can survive the template substitution in the rendered brief. Update the list body so the continuation hint is not written as a {{token}}-shaped literal; either move it to a non-substituted comment like the HTML template does or remove it entirely. Use the nearby immediate_action_* placeholders in the template to locate the affected section.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@plugins/auth0/skills/auth0-checkmate/references/report-template.md`:
- Around line 115-118: The list in report-template.md still contains a literal
{{...}} placeholder that can survive the template substitution in the rendered
brief. Update the list body so the continuation hint is not written as a
{{token}}-shaped literal; either move it to a non-substituted comment like the
HTML template does or remove it entirely. Use the nearby immediate_action_*
placeholders in the template to locate the affected section.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 4d87555a-5b9b-47a4-95d4-cb1a798f3e5d
📒 Files selected for processing (4)
plugins/auth0/skills/auth0-checkmate/references/auth0-cli-readme.mdplugins/auth0/skills/auth0-checkmate/references/report-template.htmlplugins/auth0/skills/auth0-checkmate/references/report-template.mdplugins/auth0/skills/auth0-checkmate/scripts/render_pdf.sh
| chmod 600 "$ENV_FILE" | ||
| ``` | ||
|
|
||
| Substitute the three `<...>` placeholders before writing (the heredoc is single-quoted to prevent unintended expansion — replace with double-quoted `<<EOF` if you need to interpolate). The skill reloads this file in Phase 4 via `source`. The user does NOT need to add anything to their shell rc — but they can, for convenience: `echo 'source ~/.auth0-checkmate.env' >> ~/.zshrc` (or `~/.bashrc`). |
There was a problem hiding this comment.
The heredoc is single-quoted (no shell expansion), so the angle-bracket placeholders like <tenant_domain> will be written literally to disk if an agent doesn't substitute them first. Add an explicit note here: "Replace <tenant_domain>, <client_id>, and <client_secret> with the values from Phase 2 before writing the file."
|
|
||
| You produce a **plan-agnostic** Auth0 tenant health check for any segment — Free, self-service paid (Essentials / Professional), or Enterprise. It fuses **CheckMate-style security/config hygiene** with **use-case capability fit**, gives **two scores**, makes **tier-adaptive** recommendations, and can **optionally apply approved fixes** via the Auth0 CLI. | ||
|
|
||
| This skill **orchestrates** the `auth0-checkmate` sister skill rather than reimplementing it: CheckMate runs the tenant scan + findings **and** gathers the lightweight company/use-case context (its Phase 3) that the Capability Fit track and the use-case-based plan recommendation build on. |
There was a problem hiding this comment.
The orchestration dependency isn't explicit: this skill needs CheckMate's Phase 3 company context output before it can run its own Phase 2. Without a clear sequencing note, an agent may try to gather company context in parallel. Add something like: "Run auth0-checkmate to completion first — its Phase 3 output is the company context this skill consumes in Phase 2."
| 1. **MAU forecast** vs. the tenant's own track ceiling — [references/mau-forecast-calculator.md](references/mau-forecast-calculator.md) + [references/pricing.md](references/pricing.md). Never hardcode limits. | ||
| 2. **Enterprise-need detection FIRST** — run [references/enterprise-need-detection.md](references/enterprise-need-detection.md). It can short-circuit plan matching. | ||
| 3. **Plan matching** keyed on current plan — [references/plan-matching-logic.md](references/plan-matching-logic.md): | ||
| - **Self-service (no Enterprise need):** recommend a specific plan + **exact cost** from `pricing.md`. Suggest the **A4AA add-on only when `a4aa_fit_score ≥ 0.4` AND there are concrete agent integrations** (`integrations.length > 0` / autonomous-action workflows in the company context) — a high fit score alone, with no integrations, is NOT enough (per `plan-matching-logic.md`). The `a4aa_fit_score` here is the health check's own metric, computed from use-case + integration signals (per `data-integrity-rules.md`), not an external score. |
There was a problem hiding this comment.
is gated here (≥ 0.4) but SKILL.md never tells the agent when or how to compute it. An agent reaching this point without the value will either skip the gate or guess. Add a pointer earlier (Phase 3B is the natural place): "Compute a4aa_fit_score per data-integrity-rules.md."
| ``` | ||
| Markdown per [references/report-template.md](references/report-template.md); HTML per [references/report-template.html](references/report-template.html); PDF via: | ||
| ```bash | ||
| ${CLAUDE_SKILL_DIR}/scripts/render_pdf.sh "$HTML_PATH" "$PDF_PATH" |
There was a problem hiding this comment.
${CLAUDE_SKILL_DIR} is used here but this skill has no portability table defining it (unlike auth0-checkmate which explains it at line 57). Add a one-liner before this block so agents on other platforms know to substitute the absolute path to this skill folder.
|
|
||
| Close out: update state, append to `history.jsonl`, print applied/queued/skipped counts, suggest re-running the health check to confirm fixes. | ||
|
|
||
| State dir: `~/.claude/skills/auth0-healthcheck-all-plans/state/` (`operator.json`, `setup.json` (cache; secrets excluded), `enrichment_<domain>_<ts>.json`, `queue.json`, `history.jsonl`). Treat `setup.json` as a cache — re-validate each run. |
There was a problem hiding this comment.
The state directory path is only defined here at the bottom, but Phase 0 (line 58) already tells the agent to read state/operator.json without establishing where state/ lives. Move the state path definition to Phase 0 so the agent has the full path before it first tries to read it.
…ck-all-plans
- README.md: trim skill descriptions to single sentence (reviewer: sanchitmehtagit)
- render_pdf.sh (both skills): propagate wkhtmltopdf/weasyprint exit codes
instead of unconditional `exit 0` — caller now sees renderer failures
- auth0-healthcheck-all-plans/SKILL.md: align state dir to ~/.auth0-checkmate/state/
to match auth0-checkmate so the enrichment cache is shared between skills
- auth0-checkmate/references/report-template.md: replace {{...}} continuation
hints with HTML comments so they don't survive template substitution
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
|
||
| ### Render the PDF | ||
| ```bash | ||
| "$SKILL_DIR/scripts/render_pdf.sh" "$HTML_PATH" "$PDF_PATH" |
There was a problem hiding this comment.
$SKILL_DIR is used here but the portability table above (line 57) defines the variable as ${CLAUDE_SKILL_DIR}. An agent following the table will have ${CLAUDE_SKILL_DIR} set, not $SKILL_DIR, so this script silently uses an empty path. Either use ${CLAUDE_SKILL_DIR} consistently throughout, or add
SKILL_DIR="${CLAUDE_SKILL_DIR}" at the top of this script block.
auth0-checkmate/SKILL.md:
- Heredoc note now names all three placeholders explicitly and shows a
sed substitution example so agents don't write literal <...> to disk
- Render PDF block: use ${CLAUDE_SKILL_DIR} consistently (was $SKILL_DIR,
which silently used an empty path vs. the portability table's variable)
auth0-healthcheck-all-plans/SKILL.md:
- Add sequencing constraint: auth0-checkmate must complete Phase 3 before
this skill's Phase 2 can start (company context dependency)
- Phase 3B: add explicit instruction to compute a4aa_fit_score via
data-integrity-rules.md so the Phase 4 gate has the value it needs
- Render PDF block: add ${CLAUDE_SKILL_DIR} portability note inline
- Phase 0: move state dir definition to top of phase so agents know the
full path before the first state read on step 1
tests/ (both skills):
- Add evals.json + graders.json covering the core audit/healthcheck flows,
plan routing, state dir, and key safety invariants
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
CI failures — not a code issueBoth failing checks are caused by GitHub Actions restrictions on fork PRs, not by the skill code.
Fork PRs run with a read-only
You can verify both by checking the lint run logs — the skillsaw output there shows Happy to address any remaining code feedback — everything else from the latest review round has been pushed in |
Thanks! Please address any pending feedback. Also, I think you may need to sign the unsigned commits.We will take care of CI checks |
2 participants
Summary
Adds two customer-facing Auth0 self-audit skills to
plugins/auth0/skills/:auth0-healthcheck-all-plansA plan-aware tenant health check for any segment (Free → Enterprise). One assessment fuses CheckMate-style security/config hygiene with use-case capability fit, emits two scores (Security & Config Hygiene + Capability Fit), and gives a use-case-based plan recommendation (a specific self-service plan with cost, or an "Enterprise — contact sales" path with a prefilled brief). Can apply approved fixes via the Auth0 CLI with per-command confirmation.
auth0-checkmateAn Auth0 CheckMate tenant audit that bootstraps the Auth0 CLI + a dedicated M2M app, gathers lightweight public company context inline (so findings read in the customer's own product terms), produces a personalized Auth0 Platform Development Review (markdown + styled PDF), triages fixes into immediate vs after-upgrading, and applies approved fixes with per-command confirmation. The health check orchestrates this as its single sister skill.
Conformance
CONTRIBUTING.md/AGENTS.md: onlySKILL.mdin each skill root, helpers underscripts/, reference docs underreferences/(kebab-case).license,metadata.author,metadata.openclaw(emoji + homepage + ClawHubrequires/os/installfor theauth0CLI).uvx skillsaw --strict→ 0 errors, 0 warnings (Grade A+).Note on scope
The health check originally also orchestrated a separate
account-enrichmentskill. That skill was internal account-qualification tooling (sales fit scores, plan-fit verdicts, firmographics), so rather than publish it to this public, customer-facing repo, its customer-facing capability only — company / use-case classification and current-auth-provider detection, used to make findings relevant — was folded intoauth0-checkmateas a sanitized inline step (Phase 3). The sales-qualification scoring and firmographics are intentionally not included. The health check's plan recommendation is its own use-case + needs + pricing logic, unchanged.🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes