Add buildkite-webhook-handler lambda to ingest webhook events from Buildkite #6998
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ildkite This is used on vLLM CI for the time being Signed-off-by: Huy Do <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ |
meta-cla
bot
added
the
CLA Signed
Signed-off-by: Huy Do <[email protected]>
This was referenced Aug 13, 2025
|
optional |
|
@huydhn do you mind provide more context for what is the purpose of storing the buildkite jobs and events in our db? This would be helpful info |
Let me ask Claude to write some unit tests, the lambda code mostly comes from Claude anyway ;) |
Let me add that into the README file. The high level context here is that all Buildkite webhook events about jobs running on vLLM will be available to query in the same way as GitHub webhook events.
|
Signed-off-by: Huy Do <[email protected]>
yangw-dev
approved these changes
Aug 14, 2025
huydhn
added a commit
that referenced
this pull request
Aug 16, 2025
A follow-up of #6998 and #7001 to sync these dynamoDB tables to ClickHouse, I will need to find a permanent home for these tables instead of using `fortesting` ### Testing Actually running and ingesting data at the moment (I update the lambda manually on AWS). Also backfilling the data from DynamoDB as I have run a debug version of #6998 since mid July. ``` python dynamo2ch.py --dynamodb-table vllm-buildkite-build-events --clickhouse-table fortesting.vllm_buildkite_builds --stored-data vllm_buildkite_builds.json ``` ``` python dynamo2ch.py --dynamodb-table vllm-buildkite-job-events --clickhouse-table fortesting.vllm_buildkite_jobs --stored-data vllm_buildkite_jobs.json ``` --------- Signed-off-by: Huy Do <[email protected]>
huydhn
added a commit
that referenced
this pull request
Aug 18, 2025
…obs tables (#7001) There are some events on dynamoDB now, so I use these event to create the schema for `vllm_buildkite_builds` and `vllm_buildkite_jobs` tables on ClickHouse. There is another for `vllm_buildkite_agents` that records events from Buildkite agents, but I will add it later in a separate PR once there are some records on dynamoDB that I can use to create that schema. ### Testing Run the two `CREATE TABLE` queries on the playground database `fortesting`. The two tables are also backfilling with data from dynamoDB because I just realize that I have been leaving the ingestion lambda #6998 running since July 16th-ish cc @simon-mo --------- Signed-off-by: Huy Do <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This lambda receives webhook events from vLLM so that we can build HUD-like dashboard there. https://app.hex.tech/533fe68e-dcd8-4a52-a101-aefba762f581/app/030kdEgDv6lSlh1UPYOkWP is an early example from Simon.
Testing
I have manually created
buildkite-webhook-handler-debuglambda and writing intovllm-buildkite-*dynamo table since few weeks back.Also create a test release for the lambda at https://github.com/pytorch/test-infra/actions/runs/16928003507/job/47967490767