Skip to content

feat: Add GitHub Action to validate PR contributions #96

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amanintech merged 2 commits into from
Mar 25, 2026
Merged

feat: Add GitHub Action to validate PR contributions #96

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
02d0384
feat: Add GitHub Action to validate PR contributions for AgentKits
amanintech Mar 23, 2026
0301756
Apply suggestions from code review
d-pamneja Mar 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
319 changes: 319 additions & 0 deletions .github/workflows/validate-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,319 @@
name: Validate PR Contribution

on:
pull_request:
types: [opened, edited, synchronize]

jobs:
validate:
if: startsWith(github.event.pull_request.title, 'feat:')
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Fetch base branch
run: git fetch origin ${{ github.event.pull_request.base.ref }}

- name: Validate contribution structure
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
set -euo pipefail

ERRORS=()
WARNINGS=()
NEW_PROJECTS=()
EXISTING_MODIFIED=()

# --- A. Compute diff and classify changed files ---

MERGE_BASE=$(git merge-base "origin/$BASE_REF" HEAD)
CHANGED_FILES=$(git diff --name-only --diff-filter=ACMRD "$MERGE_BASE"...HEAD || true)

if [ -z "$CHANGED_FILES" ]; then
echo "No changed files detected."
echo "## PR Validation Results" >> "$GITHUB_STEP_SUMMARY"
echo "No contribution files detected in this PR." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi

# Known exceptions to skip
SKIP_PATHS=("templates/indexation" "kits/agentic/stock-analysis")

should_skip() {
local path="$1"
for skip in "${SKIP_PATHS[@]}"; do
if [ "$path" = "$skip" ]; then
return 0
fi
done
return 1
}

# Extract project path from a file path
get_project_path() {
local file="$1"
if [[ "$file" == kits/* ]]; then
# kits/<category>/<kit-name>/... → 3 levels
echo "$file" | cut -d/ -f1-3
elif [[ "$file" == bundles/* ]]; then
# bundles/<bundle-name>/... → 2 levels
# Handle nested bundles like bundles/sample/chatbot
local second
second=$(echo "$file" | cut -d/ -f2)
if [ "$second" = "sample" ]; then
echo "$file" | cut -d/ -f1-3
else
echo "$file" | cut -d/ -f1-2
fi
elif [[ "$file" == templates/* ]]; then
# templates/<template-name>/... → 2 levels
echo "$file" | cut -d/ -f1-2
else
echo ""
fi
}

# Get contribution type from project path
get_type() {
local path="$1"
if [[ "$path" == kits/* ]]; then
echo "kit"
elif [[ "$path" == bundles/* ]]; then
echo "bundle"
elif [[ "$path" == templates/* ]]; then
echo "template"
fi
}

# Collect unique project paths
declare -A PROJECT_MAP
OTHER_FILES=()

while IFS= read -r file; do
[ -z "$file" ] && continue
project_path=$(get_project_path "$file")
if [ -z "$project_path" ]; then
OTHER_FILES+=("$file")
else
PROJECT_MAP["$project_path"]=1
fi
done <<< "$CHANGED_FILES"

# --- B. Check 1: No edits to existing projects ---

CHECK1_PASS=true

for project_path in "${!PROJECT_MAP[@]}"; do
if should_skip "$project_path"; then
continue
fi

# Check if the project existed at the merge base
if git ls-tree --name-only "$MERGE_BASE" -- "$project_path" 2>/dev/null | grep -q .; then
EXISTING_MODIFIED+=("$project_path")
ERRORS+=("Existing project modified: $project_path — feat: PRs should only add new contributions")
CHECK1_PASS=false
else
NEW_PROJECTS+=("$project_path")
fi
done

# --- C. Check 2: Root file presence ---

CHECK2_PASS=true

for project_path in "${NEW_PROJECTS[@]}"; do
if should_skip "$project_path"; then
continue
fi

ptype=$(get_type "$project_path")

case "$ptype" in
kit)
for req in config.json README.md; do
if [ ! -f "$project_path/$req" ]; then
ERRORS+=("Missing $req in $project_path")
CHECK2_PASS=false
fi
done
if [ ! -d "$project_path/flows" ]; then
ERRORS+=("Missing flows/ directory in $project_path")
CHECK2_PASS=false
fi
;;
Comment on lines +137 to +149

@coderabbitai coderabbitai Bot Mar 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check what files existing kits actually contain to determine the baseline
echo "=== Checking file patterns in existing kits ==="
for kit_dir in kits/*/*/; do
  if [ -d "$kit_dir" ]; then
    echo "Kit: $kit_dir"
    ls -la "$kit_dir" 2>/dev/null | head -15
    echo "---"
  fi
done | head -100

Repository: Lamatic/AgentKit

Length of output: 5140

🏁 Script executed:

#!/bin/bash
# Check for package.json and .env.example across all kits
echo "=== Checking for package.json and .env.example in all kits ==="
echo ""
for kit_path in kits/*/*/; do
  if [ -d "$kit_path" ]; then
    kit_name=$(basename "$kit_path")
    has_package_json="❌"
    has_env_example="❌"
    has_config="❌"
    has_readme="❌"
    has_flows="❌"
    
    [ -f "$kit_path/package.json" ] && has_package_json="✓"
    [ -f "$kit_path/.env.example" ] && has_env_example="✓"
    [ -f "$kit_path/config.json" ] && has_config="✓"
    [ -f "$kit_path/README.md" ] && has_readme="✓"
    [ -d "$kit_path/flows" ] && has_flows="✓"
    
    echo "Kit: $kit_name"
    echo "  config.json: $has_config  README.md: $has_readme  flows/: $has_flows  .env.example: $has_env_example  package.json: $has_package_json"
  fi
done

Repository: Lamatic/AgentKit

Length of output: 1050

Add validation for .env.example and package.json in kit checks.

The workflow validates only config.json, README.md, and flows/ for kits, but documentation requires .env.example and package.json. Evidence shows kits already fail these requirements:

  • kits/assistant/grammar-extension/ is missing both .env.example and package.json
  • kits/embed/content-generation/ is missing package.json

Both pass the current workflow validation, violating documented requirements.

bundle)
for req in config.json README.md; do
if [ ! -f "$project_path/$req" ]; then
ERRORS+=("Missing $req in $project_path")
CHECK2_PASS=false
fi
done
if [ ! -d "$project_path/flows" ]; then
ERRORS+=("Missing flows/ directory in $project_path")
CHECK2_PASS=false
fi
;;
template)
for req in config.json inputs.json meta.json README.md; do
if [ ! -f "$project_path/$req" ]; then
ERRORS+=("Missing $req in $project_path")
CHECK2_PASS=false
fi
done
;;
esac
done

# --- D. Check 3: Flow folder validation ---

CHECK3_PASS=true
FLOW_REQUIRED=(config.json inputs.json meta.json README.md)

for project_path in "${NEW_PROJECTS[@]}"; do
if should_skip "$project_path"; then
continue
fi

ptype=$(get_type "$project_path")

# Only kits and bundles require flows/ validation
if [ "$ptype" = "template" ]; then
continue
fi

if [ ! -d "$project_path/flows" ]; then
# Already caught in Check 2
continue
fi

# Check at least one flow exists
flow_count=0
for flow_dir in "$project_path/flows"/*/; do
[ -d "$flow_dir" ] || continue
flow_count=$((flow_count + 1))

for req in "${FLOW_REQUIRED[@]}"; do
if [ ! -f "$flow_dir$req" ]; then
ERRORS+=("Missing $req in $flow_dir")
CHECK3_PASS=false
fi
done
done

if [ "$flow_count" -eq 0 ]; then
ERRORS+=("No flow subdirectories found in $project_path/flows/ — at least one flow is required")
CHECK3_PASS=false
fi
done

# --- E. Check 4: Warn on changes outside contribution dirs ---

CHECK4_WARN=false
if [ ${#OTHER_FILES[@]} -gt 0 ]; then
CHECK4_WARN=true
for f in "${OTHER_FILES[@]}"; do
WARNINGS+=("File outside kits/bundles/templates modified: $f")
done
fi

# --- F. Output results to job summary ---

{
echo "## PR Validation Results"
echo ""

if [ ${#NEW_PROJECTS[@]} -gt 0 ]; then
echo "### New Contributions Detected"
for p in "${NEW_PROJECTS[@]}"; do
ptype=$(get_type "$p")
echo "- **${ptype^}**: \`$p\`"
done
echo ""
fi

if [ ${#EXISTING_MODIFIED[@]} -gt 0 ]; then
echo "### Existing Projects Modified (not allowed in feat: PRs)"
for p in "${EXISTING_MODIFIED[@]}"; do
echo "- \`$p\`"
done
echo ""
fi

echo "### Check Results"
echo ""
echo "| Check | Status |"
echo "|-------|--------|"

if [ "$CHECK1_PASS" = true ]; then
echo "| No edits to existing projects | :white_check_mark: Pass |"
else
echo "| No edits to existing projects | :x: Fail |"
fi

if [ "$CHECK2_PASS" = true ]; then
echo "| Required root files present | :white_check_mark: Pass |"
else
echo "| Required root files present | :x: Fail |"
fi

if [ "$CHECK3_PASS" = true ]; then
echo "| Flow folder structure valid | :white_check_mark: Pass |"
else
echo "| Flow folder structure valid | :x: Fail |"
fi

if [ "$CHECK4_WARN" = true ]; then
echo "| No changes outside contribution dirs | :warning: Warning |"
else
echo "| No changes outside contribution dirs | :white_check_mark: Pass |"
fi

echo ""

if [ ${#ERRORS[@]} -gt 0 ]; then
echo "### Errors"
echo ""
for err in "${ERRORS[@]}"; do
echo "- :x: $err"
done
echo ""
fi

if [ ${#WARNINGS[@]} -gt 0 ]; then
echo "### Warnings"
echo ""
for warn in "${WARNINGS[@]}"; do
echo "- :warning: $warn"
done
echo ""
fi

if [ ${#ERRORS[@]} -eq 0 ]; then
echo "---"
echo ":tada: All checks passed! This contribution follows the AgentKit structure."
else
echo "---"
echo ":stop_sign: Please fix the errors above before this PR can be merged."
echo ""
echo "Refer to [CONTRIBUTING.md](./CONTRIBUTING.md) and [CLAUDE.md](./CLAUDE.md) for the expected folder structure."
fi
} >> "$GITHUB_STEP_SUMMARY"

# Print to logs as well
if [ ${#ERRORS[@]} -gt 0 ]; then
echo ""
echo "=== VALIDATION FAILED ==="
for err in "${ERRORS[@]}"; do
echo "::error::$err"
done
exit 1
fi

echo ""
echo "=== ALL CHECKS PASSED ==="
Loading