Skip to content

Cache the result of DaskManager.normalize_chunks #9897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Cache the result of DaskManager.normalize_chunks #9897

wants to merge 2 commits into from

Conversation

dcherian
Copy link
Contributor

@dcherian dcherian commented Dec 16, 2024 •
edited
Loading

This is only used with the backends codepath, where the inputs
are guaranteed to be tuples. By contrast, dask.array.normalize_chunks
accepts dicts as inputs and so, is harder to cache transparently

xref #8902
xref #1525

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

@dcherian
Cache the result of DaskManager.normalize_chunks
0e34528 
This is only used with the backends codepath, where the inputs
are guaranteed to be tuples. By contrast, `dask.array.normalize_chunks`
accepts dicts as inputs and so, is harder to cache transparently
@dcherian
Try using uuids
de010ea 
xref pydata#8902
xref pydata#1525
@TomNicholas TomNicholas added topic-dask topic-chunked-arrays Managing different chunked backends, e.g. dask labels Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
topic-chunked-arrays Managing different chunked backends, e.g. dask topic-dask
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dcherian @TomNicholas