Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Hybrid executors for running task logs #45971

Conversation

dmelchor-stripe
Copy link

@dmelchor-stripe dmelchor-stripe commented Jan 23, 2025

Using Hybrid executors where one of them is a Kubernetes-based executor (but not the first one) breaks live logs since we only try to fetch k8s pods logs if the default executor happens to be the Kubernetes executor. This PR fixes it.

Note: it's my first PR and I probably don't have time to set up the whole testing environment. If you consider this change requires thorough testing please FFT to close this and revisit at a further date (just wanted to flag this)

Copy link

boring-cyborg bot commented Jan 23, 2025

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

@dmelchor-stripe dmelchor-stripe force-pushed the dmelchor/hybrid-executors-k8s-logs branch from f907834 to 1e2b10e Compare January 23, 2025 14:55
@eladkal eladkal requested a review from o-nikolas January 23, 2025 15:15
@dmelchor-stripe dmelchor-stripe force-pushed the dmelchor/hybrid-executors-k8s-logs branch from 75d0396 to 4c5c47f Compare January 23, 2025 16:50
@o-nikolas
Copy link
Contributor

We already have an extensively tested and iterated on fix for the same issue here: #45631

I'll close this one in favour of the existing one. Thanks for the contribution though!

@o-nikolas o-nikolas closed this Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants