Component discovery: migration of addons to support new filemask property

[surli]

Hi everyone,

I have an instance of Weblate where I setup the discovery addons a while ago, before the filemask property, that allows to find component with only a source translation file, was introduced.
I now want to update my addon configuration to use the filemask property, but I'm not entirely sure how to do it and I'm getting some issues.

So right now I have the following setup:

Key	Value
match	(?P<originalHierarchy>.+/)(?P<component>[^/]*)/src/main/resources/ApplicationResources_(?P<language>[^/.]*).properties
file format	XWiki Java Properties
name template	{{component}}
base file template	{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources.properties
new base template	{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources_{{ language }}.properties
file mask

My understanding is that I should use the following configuration now:

Key	Value
match	(?P<originalHierarchy>.+/)(?P<component>[^/]*)/src/main/resources/ApplicationResources.properties
file format	XWiki Java Properties
name template	{{component}}
base file template	{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources.properties
new base template	??
file mask	{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources_*.properties

Now I have two issues:

1. I have no idea how I should setup the new base template property, since I cannot use anymore the {{ language }}?
2. When I try the setup above with an empty value for the new base template property (which is unrelated I guess), I can't see the list of matching components to validate I'm systematically getting:

Note that our repository contains hundreds of translated modules, so maybe it is to be expected, but nevertheless it doesn't help to know if my config is good and I'm a bit reluctant to try as I'm afraid to get tons of new components wrongly found in case of mistake...

[prathmesh235]

I think the migration is basically correct, but there is one important distinction with new_base_template.

Once you set a file mask, the discovery regex should match the monolingual base/new-base file, so removing the language group is expected. The language-specific files are then discovered through the file mask.

For your layout, I would use:

match:
(?P<originalHierarchy>.+/)(?P<component>[^/]*)/src/main/resources/ApplicationResources.properties

base file template:
{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources.properties

file mask:
{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources_*.properties

For new_base_template, I would leave it empty if you do not have a separate source/template file that should be used when creating a new translation. new_base_template is the filename used as the base for creating new translations; it is not a replacement for the old {{ language }} value.

The other important point is that, with a file mask configured, Weblate uses the base/new-base file for discovery and then applies the file mask to find the language variants. So the fact that {{ language }} is no longer available in the discovery regex is intentional.

I would also add a language filter if your repository contains many unrelated ApplicationResources_*.properties files. That can make the discovery scan much safer and reduce accidental matches.

So I would test the configuration with:

match:
(?P<originalHierarchy>.+/)(?P<component>[^/]*)/src/main/resources/ApplicationResources.properties

file format:
XWiki Java Properties

name template:
{{ component }}

base file template:
{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources.properties

new base template:
(empty, unless you have a separate new-translation base file)

file mask:
{{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources_*.properties

The current Weblate documentation also describes this exact migration pattern: when filemask is set, omit the language group from the match and use a wildcard plus {{ component }} in the file mask.

Before enabling removal/creation across the whole repository, I would test the regex/file-mask combination against a small representative directory first, since your repository has hundreds of translated modules.

[nijel]

Just to extend the previous reply: I doubt that including language in the new base was ever a good idea. It was probably picking a random language. {{ originalHierarchy }}{{ component }}/src/main/resources/ApplicationResources.properties should work well as new base - using the the monolingual template is usually the best approach.