Skip to content

Conversation

rafaqz
Copy link
Owner

@rafaqz rafaqz commented Jul 7, 2025

Not sure how we never hit this before, but Center Contains is broken for DateTime. Needs a test

Oh, its because its annoyingly hard to define what the step size Month or Year (with leap years) means exactly when it doesn't start at the start of the period.

We could try using each the weigted mean of the overlapped months, with the weight coming from the distance the center point is from either one? Just have to make sure it works out with no gaps.

The other reason this has never been a real problem is the only time you get Center time is in netcdf, and there is always an Explicit bounds matrix in that case, and that already works fine.

But @tiemvanderdeure maybe you have some thoughts on this

Copy link

codecov bot commented Jul 7, 2025

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 84.41%. Comparing base (bc6a4c8) to head (3b41292).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/Lookups/selector.jl 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1037      +/-   ##
==========================================
+ Coverage   84.03%   84.41%   +0.37%     
==========================================
  Files          52       52              
  Lines        5136     5139       +3     
==========================================
+ Hits         4316     4338      +22     
+ Misses        820      801      -19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tiemvanderdeure
Copy link
Collaborator

tiemvanderdeure commented Jul 7, 2025

What bug does this fix exactly? I'm pretty sure I've also run into rounding errors with datetime dimensions but I'm not sure if I ever made an issue

EDIT: figured it out - you get this when trying to shiftlocus

@tiemvanderdeure
Copy link
Collaborator

Still getting an error on this branch when selecting with a Date and the lookup is DateTime

using Dates, DimensionalData, DimensionalData.Lookups
tidim = Ti(Sampled(DateTime(2000):Month(1):DateTime(2001); sampling = Intervals())) |> DimensionalData.format
tidim2 = shiftlocus(Center(), tidim)
da = rand(tidim2)
da[Ti = Contains(DateTime(2000, 5, 1))] # Fine
da[Ti = Contains(Date(2000, 5, 1))] # errors
ERROR: MethodError: no method matching -(::DateTime, ::Date)
The function `-` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  -(::DateTime, ::Year)
   @ Dates ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Dates/src/arithmetic.jl:33
  -(::DateTime, ::Month)
   @ Dates ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Dates/src/arithmetic.jl:62
  -(::DateTime, ::Quarter)
   @ Dates ~/.julia/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Dates/src/arithmetic.jl:78
  ...

Stacktrace:
  [1] #contains#187
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:463 [inlined]
  [2] contains
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:456 [inlined]
  [3] contains(sampling::Intervals{…}, l::Sampled{…}, sel::Contains{…}; err::DimensionalData.Dimensions.Lookups._True)
    @ DimensionalData.Dimensions.Lookups ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:414
  [4] contains
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:412 [inlined]
  [5] #contains#73
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:393 [inlined]
  [6] contains
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:391 [inlined]
  [7] _selectindices
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:1097 [inlined]
  [8] selectindices
    @ ~/.julia/dev/DimensionalData/src/Lookups/selector.jl:1075 [inlined]
  [9] _dims2indices
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:144 [inlined]
 [10] _dims2indices
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:136 [inlined]
 [11] dims2indices
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:31 [inlined]
 [12] macro expansion
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:89 [inlined]
 [13] split_alignments
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:89 [inlined]
 [14] split_alignments
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:87 [inlined]
 [15] dims2indices
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:56 [inlined]
 [16] dims2indices
    @ ~/.julia/dev/DimensionalData/src/Dimensions/indexing.jl:32 [inlined]
 [17] _dim_getindex
    @ ~/.julia/dev/DimensionalData/src/array/indexing.jl:106 [inlined]
 [18] #getindex#176
    @ ~/.julia/dev/DimensionalData/src/array/indexing.jl:75 [inlined]
 [19] top-level scope
    @ ~/juliadev/DimensionalData/test/playground.jl:31
Some type information was truncated. Use `show(err)` to see complete types.

@rafaqz
Copy link
Owner Author

rafaqz commented Jul 7, 2025

The bug is that some selectors dont work on Regular Center Intervals lookups with DateTime becuase its not clear how to find the intervalbounds exactly, because the step size of e.g. Month isn't a fixed amount.

Pracitcally the bug is that trying e.g. Contains or Between currently just gives an error.

(and no the PR isn't solved because I'm not really sure how to yet... one option could be just failing for Month/Year and getting the rest right as they are exact quantities? I dont know really

@tiemvanderdeure
Copy link
Collaborator

For DateTime an option is that the intervalbounds are just the middle of x and x+span

One problem is that in Julia things like dividing by 2, mean, and middle are not defined on dates/datetimes. I don't think we can get around breaking this convention (at least internally).

There is some discussion here JuliaLang/julia#54542

@rafaqz
Copy link
Owner Author

rafaqz commented Jul 7, 2025

Yeah. But step sizes of days, hours and seconds are pretty well defined so we could at least make them work everywhere without ambiguity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants