Skip to content

Commit

Permalink
test
Browse files Browse the repository at this point in the history
  • Loading branch information
jerrykingxyz committed Dec 27, 2024
1 parent 74bf045 commit 9ae7966
Showing 1 changed file with 26 additions and 14 deletions.
40 changes: 26 additions & 14 deletions .github/workflows/ecosystem-benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
push:
branches:
- main
- jerry/test
paths-ignore:
- "**/*.md"
- "website/**"
Expand All @@ -29,7 +30,7 @@ jobs:
target: x86_64-unknown-linux-gnu
native: ${{ needs.get-runner-labels.outputs.LINUX_RUNNER_LABELS == '"ubuntu-22.04"' }}
runner: ${{ needs.get-runner-labels.outputs.LINUX_RUNNER_LABELS }}
ref: ${{ github.event_name == 'workflow_dispatch' && format('pull/{0}/head', inputs.pr) || "refs/heads/main" }}
ref: ${{ github.event_name == 'workflow_dispatch' && format('pull/{0}/head', inputs.pr) || github.sha }}
test: false
bench: false

Expand All @@ -41,39 +42,41 @@ jobs:
- id: create-comment
uses: actions/github-script@v7
with:
github-token: ${{ GITHUB_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
result-encoding: string
script: |
const url = `${context.serverUrl}//${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`
const urlLink = `[Open](${url})`
const body = `⏳ Triggered benchmark: ${urlLink}`
if (context.eventName === 'workflow_dispatch') {
const { data: comment } = await github.rest.issues.createComment({
issue_number: context.payload.inputs.pr,
owner: context.repo.owner,
repo: 'rspack',
body: `⏳ Triggered benchmark: ${urlLink}`
repo: context.repo.repo,
body
})
return comment.id
}
const { data: comment } = await github.rest.repos.createCommitComment({
commit_sha: context.sha,
owner: context.repo.owner,
repo: 'rspack',
repo: context.repo.repo,
body
})
return comment.id
bench:
needs: [build]
runs-on: [self-hosted]
outputs:
diff-result: ${{ steps.run-benchmark.outputs.diff-result }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
ref: ${{ github.event_name == 'workflow_dispatch' && format('pull/{0}/head', inputs.pr) || github.sha }}

- name: Clean
uses: ./.github/actions/clean
Expand Down Expand Up @@ -104,7 +107,7 @@ jobs:
git clone --single-branch --depth 1 https://github.com/web-infra-dev/rspack-ecosystem-benchmark.git
cd rspack-ecosystem-benchmark
pnpm i
RSPACK_DIR=RSPACK_DIR node bin/cli.js bench
RSPACK_DIR="$RSPACK_DIR" node bin/cli.js bench
result=$(node bin/cli.js compare --base latest --current current)
echo "$result"
echo "diff-result=${result//$'\n'/'@@'}" >> $GITHUB_OUTPUT
Expand All @@ -116,7 +119,7 @@ jobs:
steps:
- uses: actions/github-script@v7
with:
github-token: ${{ secrets.RSPACK_BOT_ACCESS_TOKEN }}
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const diffResult = `${{ needs.bench.outputs.diff-result }}`
Expand All @@ -133,12 +136,21 @@ jobs:
${result}
`
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: 'rspack',
comment_id: `${{ needs.create-comment.outputs.comment-id }}`,
body
})
if (context.eventName === 'workflow_dispatch') {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: `${{ needs.create-comment.outputs.comment-id }}`,
body
})
} else {
await github.rest.repos.updateCommitComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: `${{ needs.create-comment.outputs.comment-id }}`,
body,
});
}
if (result.includes("Threshold exceeded")) {
console.log("Some benchmark cases exceed the threshold, please visit the previous step for more information");
Expand Down

1 comment on commit 9ae7966

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented on 9ae7966 Dec 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Benchmark detail: Open

Name Base (2024-12-27 170b877) Current Change
10000_big_production-mode_disable-minimize + exec 37.9 s ± 592 ms 38.3 s ± 402 ms +1.20 %
10000_development-mode + exec 1.86 s ± 25 ms 1.82 s ± 32 ms -2.27 %
10000_development-mode_hmr + exec 682 ms ± 12 ms 672 ms ± 5.5 ms -1.46 %
10000_production-mode + exec 2.5 s ± 32 ms 2.43 s ± 16 ms -2.72 %
arco-pro_development-mode + exec 1.76 s ± 46 ms 1.77 s ± 127 ms +0.41 %
arco-pro_development-mode_hmr + exec 377 ms ± 0.82 ms 376 ms ± 0.81 ms -0.12 %
arco-pro_production-mode + exec 3.62 s ± 91 ms 3.64 s ± 129 ms +0.57 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 3.67 s ± 53 ms 3.71 s ± 137 ms +0.97 %
arco-pro_production-mode_traverse-chunk-modules + exec 3.62 s ± 114 ms 3.59 s ± 71 ms -0.85 %
threejs_development-mode_10x + exec 1.51 s ± 11 ms 1.52 s ± 25 ms +0.45 %
threejs_development-mode_10x_hmr + exec 790 ms ± 33 ms 779 ms ± 17 ms -1.44 %
threejs_production-mode_10x + exec 5.4 s ± 129 ms 5.36 s ± 76 ms -0.69 %
10000_big_production-mode_disable-minimize + rss memory 9505 MiB ± 278 MiB 9522 MiB ± 35.7 MiB +0.18 %
10000_development-mode + rss memory 653 MiB ± 10.9 MiB 697 MiB ± 28.5 MiB +6.84 %
10000_development-mode_hmr + rss memory 1451 MiB ± 170 MiB 1569 MiB ± 281 MiB +8.20 %
10000_production-mode + rss memory 628 MiB ± 25.7 MiB 686 MiB ± 34.2 MiB +9.08 %
arco-pro_development-mode + rss memory 578 MiB ± 26.7 MiB 587 MiB ± 28.3 MiB +1.59 %
arco-pro_development-mode_hmr + rss memory 610 MiB ± 104 MiB 610 MiB ± 75.9 MiB -0.02 %
arco-pro_production-mode + rss memory 731 MiB ± 44.6 MiB 751 MiB ± 38.8 MiB +2.77 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 745 MiB ± 71.9 MiB 760 MiB ± 55.6 MiB +1.97 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 759 MiB ± 30.2 MiB 752 MiB ± 50.3 MiB -0.91 %
threejs_development-mode_10x + rss memory 594 MiB ± 12.2 MiB 645 MiB ± 18.6 MiB +8.48 %
threejs_development-mode_10x_hmr + rss memory 1171 MiB ± 171 MiB 1202 MiB ± 70.2 MiB +2.67 %
threejs_production-mode_10x + rss memory 873 MiB ± 52.4 MiB 912 MiB ± 29.2 MiB +4.49 %

Please sign in to comment.