Add support for localizing LoopBuilders

[ctrueden]

Following on to @etarena's question in the Gitter channel the other day ("Is there a way to use LoopBuilder (or something similar) to loop multiple things while knowing their position?"), I looked into what it would take to add support for LoopBuilder loops that include a Localizable.

Turns out it was pretty straightforward! Here is a very quick first cut. Comments:

• Two images only for now, as proof of concept, but adding the other arities would be trivial.
• I don't love the naming (setImagesLocalizing) but just picked something to move forward. I don't feel strongly about it.
• The unit tests are just to show it works—I expect @maarzt would want to do them differently.
• I didn't add support for cursor-based localization because localizing cursors never pass the allCursorsAreFast check and so LoopBuilder will fall back to random accesses anyway.
• Looking at the current LoopBuilder design, mostly it is quite elegant! But I got a bad feeling about the fact that the *ConsumerRunnable classes hardcode Sampler arguments and calls to Sampler.get() when they could potentially be either A) more general or B) more specifically named to indicate that.

Feedback on whether this is a thing we want to have? Did I miss an already existing way to do this? It would have been helpful on a recent project with @etarena where we had to loop several images in sync, while needing to localize them at each point...

[tpietzsch]

I can think of two alternatives that I would prefer to make this more general, that could be even implemented both.

1.)
Make a RandomAccessibleInterval<Localizable> that just reports the coordinates as values. (Maybe that already exists, but I couldn't find it quickly. net.imglib2.realtransform.PositionRandomAccessible is something in that general direction, but not exactly).

We could add a convenience method maybe to Intervals to create that. Then you could pass it as an additional image to the "normal" LoopBuilder.

Instead of

LoopBuilder.setImagesLocalizing( imageA, imageB )
        .forEachPixel( (l, a, b) -> {...} );

it would be

LoopBuilder.setImages( Intervals.positions(imageA), imageA, imageB )
        .forEachPixel( (l, a, b) -> {...} );

Not much more to write, but now you can choose whether to provide coordinates of imageA, imageB, both, or something else...

2.)
Pass the cursors/randomaccesses to the forEachPixel function. Instead of consuming (A, B) in the forEachPixel, consume (LocalizableSampler<A>, LocalizableSampler<B>).
Instead of

LoopBuilder.setImagesLocalizing( imageA, imageB )
        .forEachPixel( (l, a, b) -> {
            ...
        } );

it would be

LoopBuilder.setImages( imageA, imageB )
        .forEachPixelLocalizing( (ca, cb) -> {
            Localizable l = ca;
            A a = ca.get();
            B b = cb.get();
            ...
        } );

[ctrueden]

Thanks @tpietzsch for your comments!

(1) is a good idea, and already exists. See Localizables.randomAccessibleInterval(Interval).

I really like (2), and had considered doing it that way when starting this work. But I got hung up on the fact that the current generic typing (LoopBuilder< T >) does not make it easy to write an alternative forEachPixelLocalizing loop that accepts a different action type than T.

I have now updated the PR to work as you describe in (2). Support for all six arities is included, so the commit is no longer a WIP. I overcame the above typing issue by introducing a second generic parameter LT on LoopBuilder for its "localizing action type". At the class level, T and LT have no relation, but the setImages methods always return objects like LoopBuilder< BiConsumer< A, B >, BiConsumer< RandomAccess< A >, RandomAccess< B > > >.

I considered generalizing this to LoopBuilder< BiConsumer< A, B >, BiConsumer< LA, LB > > > where LA extends Sampler< A > & Localizable and LB extends Sampler< B > & Localizable (or a new LocalizableSampler combo interface as you suggested), but those expressions become very long at higher arities, and LoopBuilder only works with RandomAccessibleInterval anyway so in practice I think it's always fine to promise RandomAccess objects explicitly. One concern is that we might not want the accessors to be explicitly Positionable—makes it seductively easy to reposition accessors inside the loop—but who knows, maybe someone finds it useful to do that, e.g. for implementing a neighborhood operation.

now you can choose whether to provide coordinates of imageA, imageB, both, or something else...

Under what conditions would a different coordinate be provided for imageA versus imageB? Aren't they always kept in sync? Regardless, having direct access to the RandomAccess of each image is nice if in fact you do want to start moving them around...

[ctrueden]

I've now added arities 7 through 10. @etarena and I were actually trying to loop over 8 synced images and ran out of LoopBuilder. This PR is ready for review.

[tpietzsch]

I considered generalizing this to LoopBuilder< BiConsumer< A, B >, BiConsumer< LA, LB > > > where LA extends Sampler< A > & Localizable and LB extends Sampler< B > & Localizable (or a new LocalizableSampler combo interface as you suggested), but those expressions become very long at higher arities, and LoopBuilder only works with RandomAccessibleInterval anyway

I think, LoopBuilder uses Cursor if they are available and expected to be faster, e.g., iterating over two ArrayImgs (and if not, we should remain open to that option). So I would go for the Sampler< A > & Localizable.

Under what conditions would a different coordinate be provided for imageA versus imageB?

LoopBuilder doesn't require the inputs to have matching coordinates, only to be of the same size.
Example would be to copy a small image into a (non-zero-min) region of a bigger image.

[tpietzsch]

already exists. See Localizables.randomAccessibleInterval(Interval).

oh oh, looks like I'm losing track of things ... :-)

[ctrueden]

I think, LoopBuilder uses Cursor if they are available and expected to be faster, e.g., iterating over two ArrayImgs (and if not, we should remain open to that option). So I would go for the Sampler< A > & Localizable.

I changed it: 107342e

This will prevent people from being able to reposition the samplers as easily since the static typing will not expose that they are Positionable. But it does leave the door open for cursor-based localizable loops in future. Note though that currently, localizing cursors are never used: they don't pass the allCursorsAreFast type check internally.

[ctrueden]

@tpietzsch Are we OK to merge it?

[tpietzsch]

I would wait for @maarzt to review, if it is not urgent.

[ctrueden]

I would wait for @maarzt to review, if it is not urgent.

Sure. Though he'd be within his rights to demand a quid pro quo for scijava/scijava-ui-swing#43 😆

[maarzt]

I like the solution (1), that @tpietzsch proposed earlier. It solves many potential use cases and is easy to understand. Plus, the code is very readable:

LoopBuilder.setImages( Intervals.positions(imageA), imageA, imageB )
        .forEachPixel( (position, a, b) -> {...} );

We would only need to add the Intervals.positions(Interval) method, which forwards to the cryptically named method Localizables.randomAccessibleInterval(Interval).

The implementation of the forEachPixelLocalizing method adds many lines of code, with a high degree of code duplication. I fear that adding these lines of code makes LoopBuilder harder to maintain.

[maarzt]

Regarding the second problem with more than six images. @etadobson I don't know the actual use case but I guess many of the 7 images are similar. It might be feasible to use

collapsedImage = Views.collapse(Views.stack(image1, ...., image6))

to merge the similar images into one image. Or if you are working with neighborhoods DiamondShape.neighborhoodRandomAccessible() could be a solution.

[maarzt]

The implementation of LoopBuilder has already many lines of code. I fear that adding these 500+ lines of code reduces the maintainability. Let us go for this shorter solution #279.

[maarzt]

Closed in favor of #279