feat: added Qodo Quality Automation Agent for (#QodoAgentChallenge)

[Mohit5Upadhyay]

User description

Description

This PR introduces the Qodo Quality Automation Agent, developed for Quality Automation under the #QodoAgentChallenge.
The agent delivers end-to-end automation for code quality, security, and compliance checks across the entire SDLC. It ensures teams can maintain consistent code standards, enforce security policies, and integrate quality gates seamlessly into their CI/CD pipelines.

• A new agent: agents/qodo-quality-automation-agent/
• This agent provides intelligent, multi-stage quality gates and comprehensive reporting — both in local and CI/CD environments — ensuring consistent, secure, and maintainable codebases.

Multi-Stage Quality Pipeline

• Environment Detection – Auto-detect local vs CI and setup thresholds.
• Code Review & Static Analysis – Identify vulnerabilities and performance issues.
• Security Scanning – Detect CVEs and secrets.
• License Compliance – Validate dependency licenses.
• Dependency Health – Assess package reliability and maintenance.
• Test Coverage – Run tests, compute metrics, and enforce coverage.
• Code Quality Metrics – Analyze maintainability, duplication, and debt.
• Reporting & Integration – Generate structured reports and GitHub comments.

Various Tools and MCP

• GitHub – Fetch PR and repo data using GITHUB_TOKEN, post quality reports as comments, and set status checks.
• qodo_merge – For complex code analysis and review.
• Filesystem – Write structured reports (JSON, Markdown, HTML) to the report/ directory.
• Shell – Execute project tests and gather coverage metrics.
• Git – Retrieve commit diffs and target branch comparisons.

Generate Output Reports

1. report/quality-metrics.json
2. report/key-findings.md
3. report/actions-required.md

Why This Change Is Needed

Maintaining consistent code quality and security in large projects is time-consuming and error-prone when done manually.

This agent automates the process, providing:

1. Consistency – Standardized checks and quality gates across all repositories.
2. Security – Continuous scanning for vulnerabilities and risky dependencies.
3. Efficiency – Saves developer time by automating repetitive review and compliance tasks.
4. Actionable Insights – Generates reports with clear recommendations and pass/fail outcomes.

---

PR Type

Enhancement

---

Description

• Introduces comprehensive Quality Automation Agent for multi-stage code quality checks

• Orchestrates security scanning, code review, license compliance, and dependency health assessment

• Generates structured reports (JSON, Markdown) with actionable recommendations and merge decisions

• Provides CI/CD integration with GitHub Actions workflow and PR comment automation

---

Diagram Walkthrough

flowchart LR
  A["Quality Automation Agent"] --> B["Environment Detection"]
  B --> C["Code Review & Static Analysis"]
  C --> D["Security Scanning"]
  D --> E["License Compliance"]
  E --> F["Dependency Health"]
  F --> G["Test Coverage Analysis"]
  G --> H["Code Quality Metrics"]
  H --> I["Report Generation"]
  I --> J["3 Output Reports"]
  J --> K["quality-metrics.json"]
  J --> L["key-findings.md"]
  J --> M["actions-required.md"]

Loading

File Walkthrough

	Relevant files
Documentation	README.md Complete agent documentation with configuration and examples agents/qodo-quality-automation-agent/README.md Comprehensive documentation for Quality Automation Agent with feature overview and multi-stage pipeline explanation Detailed configuration table with 12 configurable arguments (mode, coverage thresholds, license lists, etc.) Quick start examples showing local and CI/CD usage patterns with various parameter combinations Quality scoring system (High/Moderate/Low) and output report specifications with JSON/Markdown examples GitHub Actions integration example and troubleshooting resources +334/-0
Configuration changes	agent.toml Agent configuration with pipeline stages and output schema agents/qodo-quality-automation-agent/agent.toml Defines quality_automation command with comprehensive instructions for 8-stage quality pipeline execution Configures 12 arguments including mode, coverage thresholds, license lists, and output formats Specifies MCP servers configuration for GitHub integration with token-based authentication Defines tools available (qodo_merge, git, filesystem, shell, github) and execution strategy as "plan" Includes detailed JSON output schema with properties for quality scores, stage results, critical issues, and merge recommendations +427/-0  github-actions.yml GitHub Actions workflow for CI/CD integration                        agents/qodo-quality-automation-agent/examples/ci-configs/github-actions.yml Complete GitHub Actions workflow for Quality Automation Agent triggered on PR and push events Configurable workflow inputs for mode, severity threshold, and minimum coverage percentage Executes agent with dynamic parameter passing and artifact upload for quality reports Implements PR commenting with key findings and actions required from generated reports Sets commit status checks based on quality score and merge safety determination +229/-0

---

[qodo-merge-for-open-source]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Token handling risk Description: The MCP GitHub server configuration embeds an Authorization header using the environment variable ${GITHUB_PERSONAL_ACCESS_TOKEN}, which risks leaking or misconfiguring secrets since typical CI environments provide GITHUB_TOKEN instead; ensure least-privilege tokens and avoid committing secret header templates that may be auto-expanded or logged. agent.toml [265-271] Referred Code "github": { "url": "https://api.githubcopilot.com/mcp/", "headers": { "Authorization": "Bearer ${GITHUB_PERSONAL_ACCESS_TOKEN}" } } }
	Sensitive info exposure Description: The PR comment includes full contents of generated Markdown reports which may contain sensitive findings (e.g., secrets or vulnerability details), potentially exposing sensitive information in PR comments and notifications; consider redacting secrets and limiting disclosure to artifacts. github-actions.yml [131-151] Referred Code github-token: ${{ secrets.GITHUB_TOKEN }} script: | const fs = require('fs'); let commentBody = `## 🚀 Quality Automation Results **Overall Quality Score:** ${{ steps.quality-check.outputs.overall_score }}/100 **Safe to Merge:** ${{ steps.quality-check.outputs.safe_to_merge == 'true' ? '✅ Yes' : '❌ No' }} **Track 4 Winner:** Best Agent for Quality Automation `; if (fs.existsSync('report/key-findings.md')) { const keyFindings = fs.readFileSync('report/key-findings.md', 'utf8'); commentBody += `### 📋 Key Findings\n\n${keyFindings}\n\n`; } if (fs.existsSync('report/actions-required.md')) { const actionsRequired = fs.readFileSync('report/actions-required.md', 'utf8'); commentBody += `### 🔧 Actions Required\n\n${actionsRequired}\n\n`; }
Ticket Compliance
⚪	🎫 No ticket provided - [ ] Create ticket/issue <!-- /create_ticket --create_ticket=true --> </details></td></tr>
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
⚪	No custom compliance provided Follow the guide to enable custom compliance check.

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-merge-for-open-source]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Re-evaluate the LLM-centric architecture The agent's logic, defined as a large LLM prompt in agent.toml, is fragile and non-deterministic. This should be replaced with a conventional script that directly orchestrates specialized tools for each quality check. Examples: agents/qodo-quality-automation-agent/agent.toml [7-244] instructions = """ You are a comprehensive Quality Automation orchestrator responsible for ensuring code quality across multiple dimensions in the SDLC. ## Core Objective Execute a multi-stage quality automation pipeline that includes: 1. Code Review & Static Analysis 2. Security Vulnerability Scanning 3. License Compliance Verification 4. Dependency Health Assessment ... (clipped 228 lines) Solution Walkthrough: Before: # agent.toml [commands.quality_automation] instructions = """ You are a comprehensive Quality Automation orchestrator... ### STAGE 1: ENVIRONMENT DETECTION ... ### STAGE 2: CODE REVIEW & STATIC ANALYSIS ... ### STAGE 3: SECURITY SCANNING ... ### STAGE 8: REPORTING & INTEGRATION MANDATORY: Generate exactly 3 files in report/ folder... """ tools = ["qodo_merge", "git", "filesystem", "shell", "github"] execution_strategy = "plan" After: # conceptual run_pipeline.py script def main(): # Stage 1: Environment Detection (scripted) ... # Stage 2-7: Run dedicated tools security_results = run_security_tool() coverage_results = run_coverage_tool() license_results = run_license_checker() # ... etc. # Stage 8: Aggregate results and generate reports metrics = aggregate_results(security_results, coverage_results, ...) generate_json_report(metrics) generate_markdown_reports(metrics) # Enforce quality gate if not metrics["safe_to_merge"]: exit(1) Suggestion importance[1-10]: 10 __ Why: The suggestion correctly identifies a fundamental architectural flaw—relying on a non-deterministic LLM prompt in agent.toml for a critical CI quality gate—and proposes a more robust, conventional, and maintainable approach.	High
Security	Use scoped GITHUB_TOKEN for security Replace the use of GITHUB_PERSONAL_ACCESS_TOKEN with the more secure, auto-generated GITHUB_TOKEN in the mcpServers configuration to align with security best practices and the provided GitHub Actions example. agents/qodo-quality-automation-agent/agent.toml [263-272] mcpServers = """ { "github": { "url": "https://api.githubcopilot.com/mcp/", "headers": { - "Authorization": "Bearer ${GITHUB_PERSONAL_ACCESS_TOKEN}" + "Authorization": "Bearer ${GITHUB_TOKEN}" } } } """ Apply / Chat Suggestion importance[1-10]: 8 __ Why: The suggestion improves security by replacing a personal access token with the scoped GITHUB_TOKEN and fixes an inconsistency between agent.toml and the example workflow.	Medium
Possible issue	Enforce merge safety for quality gate Correct the Quality Gate logic to strictly fail the workflow when safe_to_merge is false, ensuring the gate's outcome accurately reflects the quality assessment. agents/qodo-quality-automation-agent/examples/ci-configs/github-actions.yml [213-229] - name: Quality Gate if: always() run: | SAFE_TO_MERGE="${{ steps.quality-check.outputs.safe_to_merge }}" - OVERALL_SCORE="${{ steps.quality-check.outputs.overall_score }}" - + if [ "$SAFE_TO_MERGE" = "true" ]; then - exit 0 - elif [ "$OVERALL_SCORE" -ge "60" ]; then + echo "Quality gate passed: Code is safe to merge." exit 0 else - if [ "$OVERALL_SCORE" -lt "40" ]; then - exit 1 - else - exit 0 - fi + echo "Quality gate failed: Code is not safe to merge." + exit 1 fi Apply / Chat Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a critical flaw in the quality gate logic that could allow unsafe code to pass CI, and the proposed change correctly enforces the safe_to_merge flag.	Medium
General	Remove redundant merge status field Remove the redundant quality_gates_passed field from the output schema, as safe_to_merge already serves as the primary indicator for merge readiness. agents/qodo-quality-automation-agent/agent.toml [296-420] -"quality_gates_passed": { - "type": "boolean", - "description": "Whether all critical quality gates passed" -}, "stages": { ... "safe_to_merge": { "type": "boolean", "description": "Whether the code meets all quality gates and is safe to merge" } [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies and proposes removing the redundant quality_gates_passed field, simplifying the output schema and preventing potential confusion.	Low
More