feat(proto): add task_type to TailLogsRequest for connector log routing#7313
feat(proto): add task_type to TailLogsRequest for connector log routing#7313pingsutw wants to merge 1 commit intoflyteorg:mainfrom
Conversation
Adds task_type and task_type_version to flyteidl2.dataproxy.TailLogsRequest and flyteidl2.workflow.TailLogsRequest so the dataplane operator can branch between pod logs and connector GetTaskLogs based on the task category. Signed-off-by: Kevin Su <pingsutw@apache.org>
There was a problem hiding this comment.
Pull request overview
Adds connector task-category hints to TailLogs requests so dataplane log tailing can be routed to a connector’s GetTaskLogs RPC (instead of pod logs) when applicable.
Changes:
- Added
task_typeandtask_type_versiontoflyteidl2.workflow.TailLogsRequestandflyteidl2.dataproxy.TailLogsRequest. - Regenerated Go protobuf + validation outputs for the updated protos.
- Updated the generated gateway OpenAPI (swagger) JSON for the affected services.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| flyteidl2/workflow/run_logs_service.proto | Adds task_type / task_type_version fields to the workflow TailLogsRequest. |
| flyteidl2/dataproxy/dataproxy_service.proto | Adds task_type / task_type_version fields to the dataplane TailLogsRequest. |
| gen/go/flyteidl2/workflow/run_logs_service.pb.go | Regenerated Go bindings reflecting the new TailLogsRequest fields. |
| gen/go/flyteidl2/workflow/run_logs_service.pb.validate.go | Regenerated validation stubs for the new fields. |
| gen/go/flyteidl2/dataproxy/dataproxy_service.pb.go | Regenerated Go bindings reflecting the new TailLogsRequest fields. |
| gen/go/flyteidl2/dataproxy/dataproxy_service.pb.validate.go | Regenerated validation stubs for the new fields. |
| gen/go/gateway/flyteidl2/workflow/run_logs_service.swagger.json | Regenerated OpenAPI output for workflow run logs service. |
| gen/go/gateway/flyteidl2/dataproxy/dataproxy_service.swagger.json | Regenerated OpenAPI output for dataplane service. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // +optional. Connector task type name (e.g. "airflow", "batch_job"). When set, the dataplane operator | ||
| // routes the request to the matching connector's GetTaskLogs RPC instead of pod logs. The UI populates | ||
| // this from the task template it already loaded for the run. | ||
| string task_type = 4; | ||
|
|
||
| // +optional. Connector task type version. Used together with task_type to identify a TaskCategory. | ||
| int32 task_type_version = 5; |
There was a problem hiding this comment.
These new fields update the .proto sources, but the non-Go generated bindings appear not to be regenerated/committed (e.g. gen/ts and gen/python TailLogsRequest types still only have the old fields). CI runs make buf and fails if it produces a diff (.github/workflows/check-generate.yml:126-128), so please rerun make buf (or at least regenerate TS/Python/Rust outputs) and commit the updated generated files.
| // +optional. Connector task type name (e.g. "airflow", "batch_job"). When set, the run service forwards this | ||
| // hint to the dataplane so log requests for connector tasks can be routed to the connector's GetTaskLogs RPC | ||
| // instead of pod logs. The UI populates this from the task template it already loaded for the run. | ||
| string task_type = 3; | ||
|
|
||
| // +optional. Connector task type version. Used together with task_type to identify a TaskCategory. | ||
| int32 task_type_version = 4; |
There was a problem hiding this comment.
After adding these fields, the TypeScript/Python/Rust generated bindings in this repo should be regenerated and committed as well (TailLogsRequest in gen/ts + gen/python currently doesn’t include task_type/task_type_version). CI enforces make buf produces a clean diff (.github/workflows/check-generate.yml:126-128), so please run make buf and commit the resulting generated changes.
|
Superseded by #7317 (connector endpoint on LogContext). |
Summary
task_typeandtask_type_versionfields toflyteidl2.dataproxy.TailLogsRequestandflyteidl2.workflow.TailLogsRequest.TailLogsrequest to a connector'sAsyncConnectorService.GetTaskLogsRPC instead of pod logs when the action is a connector task.Test Plan
buf generateregenerated Go bindings;go build ./...clean.Notes
This supersedes #7290 (which targeted master). Reopened against main since main is the active flyte v2 trunk.