Add torchvision.transforms.Resize interpolation and antialias.

[hiyuh]

torchvision.transforms.Resize forced nearest interpolation and no antialias, but shouln't.
Based on my understanding, original torchvision.transforms.Resize calls like;

• torchvision.transforms.Resize
  • torchvision.transforms.functional.resize
    • torchvision.transforms._functional_pil.resize
      • PIL.Image.Image.resize
    • torchvision.transforms._functional_tensor.resize
      • torch.nn.functional.interpolate

Note, this PR still keeps nearest interpolation and no antialias by default for torchvision.transforms.Resize to maximize compatibility for existing code using TorchSharp and results in incompatible to original torchvision.transforms.Resize default, however, it would be up to the upstream decision.
See also;

• https://pytorch.org/vision/main/generated/torchvision.transforms.Resize.html
• https://pytorch.org/vision/main/generated/torchvision.transforms.functional.resize.html
• https://pytorch.org/docs/stable/generated/torch.nn.functional.interpolate.html

[hiyuh]

@dotnet-policy-service agree company="HACARUS"

[alinpahontu2912]

Hey @hiyuh, can you rebase and update releasenotes?

[hiyuh]

@alinpahontu2912
done, please go ahead.

[ozanMSFT]

Instead of changing user's intended value for antialias ; I would recommend to throw an informative ArgumentException to warn user such as antialias is true, however ...

[hiyuh]

@ozanMSFT

• this silent antialias changing is just mimicking what the latest torchvision (v0.21.0) does.
  • https://github.com/pytorch/vision/blob/v0.21.0/torchvision/transforms/_functional_tensor.py#L455
• i don't see any TorchSharp's upstream preference for this case like "silent user's intended value change in Python torchvision".
  • if you still consider throwing ArgumentException is better for this case, i have no objection.

[ozanMSFT]

@hiyuh , thanks for the reference.

Yes, it seems torchvision also has similar behavior but they added a comment just in the relevant part:

        # We manually set it to False to avoid an error downstream in interpolate()
        # This behaviour is documented: the parameter is irrelevant for modes
        # that are not bilinear or bicubic. We used to raise an error here, but
        # now we don't as True is the default.

So, if we're not going with the ArgumentException:

• Could you please add a comment for the explanation in antialias = false change by adding reference to torchvision 's relevant code link.

• Could you please add an similar explanation (e.g. like below) in the /// <param name="antialias"> section?

  /// antialias value will be automatically set to `false` silently in case interpolation is not InterpolationMode.Bilinear or InterpolationMode.Bicubic

[ozanMSFT]

There might be some users who are using this call with Resize(int size, int? maxSize = null) signature.

I suggest maintaining the signature order for maxSize and placing default parameters at the end to benefit existing users.

[hiyuh]

@ozanMSFT

• tried to add compatibility handling for maxSize.
• reverted test/TorchSharpTest/TestTorchVision.cs change, to demonstrate compatibility handling.
• this produces ambiguity errors from Resize(height, width) and resize(input, height, width).
  • is it possible to resolve this?
  • or TorchSharp upstream won't care about Python's argument order?

[ozanMSFT]

Thanks for the change @hiyuh , it looks good.

Regarding your question, we definitely consider `PyTorch' 's signature and argument order; but also we should consider backward compatibility whenever it is possible/applicable.

this produces ambiguity errors from Resize(height, width) and resize(input, height, width)

I couldn't see ambiguity in this file, could you please give more details or example where this error is happening?

[ozanMSFT]

Could you please add more tests for antialias behaviors? (both for true and false scenarios)

[hiyuh]

@ozanMSFT

• i don't see any benefit to maintain test for antialias functionality in TorchSharp, since which is mostly offloaded to libtorch.
• if you still consider to maintain test for antialias functionality in TorchSharp is mandatory, i can add some ones by copying the current output as expectation.

[ozanMSFT]

@hiyuh , yes please, I kindly would like to ask you to add the test scenarios.

This is not only for the functionality but also for preventing the future accidental errors and create awareness regarding the antialias functionality.

[ozanMSFT]

I added a few more comments.

@hiyuh I have some concerns regarding the compatibility between the current TorchSharp's TorchVision and your changes. Could you specify in which version of TorchVision the antialias parameter was introduced, and whether it is compatible with the current TorchSharp's native dependencies?

[ozanMSFT]

@hiyuh ; if you can give us permission for your branch as following setting, we can also resolve some RELEASENOTES conflicts on behalf of you:

NOTE: We're planning a solution for the releasenotes conflicts.

[hiyuh]

@ozanMSFT

• regarding your review comments on several source codes, let me try to improve, so wait a moment.
• about allowing maintaner's change on pulll request, from my best knowledge, it's unavailble option if the forked branch is under organization.
  • next time, i can work under my account (not my organization) if you guys preferred.

[hiyuh]

@ozanMSFT

Could you specify in which version of TorchVision the antialias parameter was introduced, and whether it is compatible with the current TorchSharp's native dependencies?

• Python torchvision antialias has been mentioned from v0.10, and not in v0.9.
  • https://pytorch.org/vision/0.10/transforms.html?highlight=resize#torchvision.transforms.Resize
  • https://pytorch.org/vision/0.9/transforms.html?highlight=resize#torchvision.transforms.Resize
• according output from following command under pytorch/pytorch,
  i assume antialias option has been available from v1.11.0.

  git tag --contains $(git log -S "options.antialias" --reverse --pretty=format:"%H" | head -n 1) | grep -e '^v' | head -n 1

  • see also pytorch/pytorch@ce9a2f8