Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement task generators and provisional nodes. #487

Merged
merged 109 commits into from
Mar 15, 2024
Merged

Conversation

tobiasraabe
Copy link
Member

@tobiasraabe tobiasraabe commented Nov 9, 2023

Closes #467.

  • Rework DelayedPathNode to DirectoryNode if they are really the same?
  • Fix typing delayed nodes. They do not need a save method for example
  • Is Delayed the right name? Should it be Abstract or something like this
  • How well do delayed nodes play along with the data catalog?
  • Settle on name provisional or delayed or fuzzy? rename modules and variables accordingly.
  • Decide whether the only way to generate tasks should be via @task to reduce the number of interfaces and maybe because it is the future?
  • Should generator=True actually be is_generator=True.
  • Should I find another abstraction for PNode and PProvisionalNode?

@tobiasraabe tobiasraabe added this to the v0.4.3 milestone Nov 9, 2023
Copy link

codecov bot commented Nov 10, 2023

Codecov Report

Attention: Patch coverage is 97.35450% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 97.75%. Comparing base (ad3680e) to head (0d41ac6).
Report is 12 commits behind head on main.

❗ Current head 0d41ac6 differs from pull request most recent head 6e5679c. Consider uploading reports for the commit 6e5679c to get more accurate results

Files Patch % Lines
src/_pytask/delayed.py 88.88% 8 Missing ⚠️
src/_pytask/delayed_utils.py 96.96% 1 Missing ⚠️
src/_pytask/shared.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #487      +/-   ##
==========================================
- Coverage   97.80%   97.75%   -0.06%     
==========================================
  Files         106      108       +2     
  Lines        8710     9042     +332     
==========================================
+ Hits         8519     8839     +320     
- Misses        191      203      +12     
Flag Coverage Δ
end_to_end 85.02% <94.70%> (+2.75%) ⬆️
integration 40.83% <49.47%> (+0.33%) ⬆️
unit 64.98% <53.70%> (-4.71%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tobiasraabe tobiasraabe changed the title Implement delayed tasks and delayed nodes. Implement task generators and provisional nodes. Mar 9, 2024
@tobiasraabe
Copy link
Member Author

tobiasraabe commented Mar 9, 2024

Hi @janosg, @timmens, do you have time for some feedback? Could you read the how-to guide and flag anything about the feature that comes to your mind? Don't worry if you do not have time. Thanks a lot!

@tobiasraabe tobiasraabe merged commit b8b7f9f into main Mar 15, 2024
20 checks passed
@tobiasraabe tobiasraabe deleted the delayed-nodes branch March 15, 2024 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: Allow the creation of delayed tasks.
1 participant