Validation of adapter configuration files

[MakisH]

Today, we had a discussion on Matrix about how to validate the adapter config file and how to distribute the schema for this validation.

@IshaanDesai asked the questions for the FEniCSx adapter (related PR: precice/fenicsx-adapter#59 (review)). @uekerman @Logende @fsimonis and myself participated in the discussion.

In general, one can get the schema from this repository, using a (versioned) direct link to the schema file.

Who checks

• One could validate the file in the adapter (or other tool) itself, potentially with an option to enable/disable this check.
  • Pro: fine-grained checks in the adapter, support for additional parameters.
  • Contra: every adapter needs to implement this, libraries are not available for all languages we need.
• An alternative would be to use the preCICE CLI
  • Pro: one central tool to maintain (already used for the config check)
  • Contra: additional tool (even if very common), no direct knowledge of additional fields (potentially: schema extensions for each adapter?)

Side condition: we are only talking about the schema here. The actual file might be in any language (#32), so the preCICE CLI would need to support different languages.

Maybe one could have both: an offline check (manually run the preCICE CLI) and an optional online check (every adapter checks at runtime). "Online" here means "when the simulation starts".

See also: #62

How to distribute

While fetching from the URL would be possible for a checker, this should, of course, not be required for running a simulation. All tools should generally be available offline.

One option would be to copy the schema file into each repository -> we want to avoid this to reduce maintenance work.

Another option would be installing the file when installing the adapter package (using the installation procedures of the adapter, or using the preCICE CLI).

@fsimonis suggested wrapping the schema into a mini Python package, which can then be distributed on PyPI, which he has already applied as a solution in the case-generate. Depending on the answer of "who checks", if that is "every adapter", then the question is how do we handle non-Python-based adapters?

[NiklasVin]

I understand the motivation behind explicitly checking whether the config file follows the guidelines. However, since the adapters support the config schema, they already implicitly test for the correctness of the config file while parsing it. If a faulty config file is not rejected, this should lead to other issues at some point. If users are unsure whether their config file does not follow the schema, they could consult the online tool (at least that is what I did). The question is whether the adapter should accept the config file as long as it can extract the necessary information following the schema or if it should accept the config files if and only if they follow the config schema. If the former is sufficient, I think the online tool is sufficient. If the latter makes more sense, then, in my opinion, there should be a universal solution for the explicit testing. A universal approach guarantees a single, consistent way to include validation, without exceptions, which minimizes additional complexity for adapter developers and users.
I would prefer not introducing the explicit validation as this was not necessary until now.

[fsimonis]

For python:

• Provide a python package that
  • Provides the newest schema version
  • Provides a library to
    • check a config based on the internally-provided schema and an optional user-defined schema
    • read the config and return a dict/namespace
    • reads/handles JSON/YAML/dict inputs
  • Provides a cli tool to check the above (and integrates with the precice-cli)

For non-python:

• unrealistic to have library support for all binding languages (C++ maybe the next priority)
• use the cli tool if possible