Definition of equality of dim tags

[albertz]

I just want to raise the issue that the equality is not really well defined in all cases.

We implement __eq__ and related functions. Which are somewhat restrictive currently, but less restrictive than object.__eq__ (which is only True when a is b, i.e. identity).

We also have is_equal. __eq__ reduces to is_equal with default options. The default options are the most restrictive equality. But the options allow to make it more loosely. E.g. such that a broadcast dim (dim = 1) can match to another dim. Or that static dims with the same static dim would match always.

There is also same_as, which is considered as always equal (recursively, so up to same_base).

The most restrictive variant currently falls back to the description when everything else (kind, dim) matches. Although I think this is mostly outdated and probably should be removed or deprecated. I'm not sure if this really makes sense. It does not check for the dynamic length tensor (dyn_size or dyn_size_ext) or identity.

Regarding dyn size, there is also the aspect of search beams or other different kind of batches. We consider dim tags as equal even when they belong to different batches / beams. So identity on a dyn_size tensor would also be wrong, and we have more loose equality. Also, we can have even the opposite, that two different dim tags make use of the same dyn size (at least in principle; while we still have get_tag_from_size_tensor, this is not really the case, but this will be resolved eventually).

Maybe the current logic is already all fine.

This might become more relevant when we rely more heavily on dim tags and their equality. E.g. for checking which axes should be shared (in GatherLayer, also see discussion in #625), or for a single reduce argument for DotLayer (#636), or once we enforce unique dim tags (#632).

In almost all cases, static dims will get different dim tags. Because most layers just specify n_out and that introduces a new dim tag (#597). Although we still consider them as equal via Data.get_common_data, which uses the is_equal options treat_feature_as_spatial=True, allow_same_spatial_dim=True, broadcast_matches=True.