consolidating potentially redundant tests

[jeromebarre]

I am seeing a lot of redundancies between:
test/aq/testinput
and
test/monarch/testinput

Can this be possibly consolidated and reduce the number of possibly duplicated tests?

Thanks!

[jeromebarre]

@benjaminmenetrier I do not seem to have permissions to assign @andreapiacentini, but needs to be assigned here.

Thank you

[andreapiacentini]

The two suites of tests are going through the same basic bricks for building a 4DEnsVar for different grids and field representations. They are meant to be "almost" identical test suites on two (aq = GEMS 01 regular lon lat regional grid, monarch = BSC MONARCH on regional rotated grid and when global data will be produced, three, with monglob for the MONARCH global grid) model configurations.
They also serve as a pedagogical introduction to VIND and JEDI yaml for the BSC staff. When they'll be familiar we could cut them down. But I don't see them as more redundant than reg/testinput vs. glb/testinput.

Anyway, at ecbuild time, the

        -DENABLE_MONARCH_TESTS="ON" \
        -DENABLE_AQ_TESTS="ON" \
        -DENABLE_MONGLOB_TESTS="OFF" \

cmake macros allow for selective activation of the tests.

[jeromebarre]

Looked like to me that aq and monarch are two regional configurations with slight differences... thererfore issuing that ticket.
Global vs regional tests are maybe a different issue.

In genertal, I am just worried as time goes on and that more groups are using vind the number of tests will grow too far too fast...
This is what happened with some interfaces and basically users put their tests in but the cost of maintenance of all this grew too far and a lot of time was spent to track down usefulness of tests and reducing their numbers.

Vind has the potential to be used by many groups but if every group is putting a large amount of test that will not go well.

I think it will be wise in the future to try to keep the tests down to a level that could be maintainable.
For the application specific yamls and tutorial we could discuss whether or not this fits into this repository or somewhere else.

I am just speaking from past experience and as another user of vind, I would be happy to help on trying to keep this as light as possible.

[andreapiacentini]

aq is a regular lon lat, therefore using NICAS for ensemble localization. Monarch is a rotated pole, described with an atlas projection and using FastLAM for localization. Therefore, there are some differences and some tests for each configuration have to be kept.
I entirely agree that we could discuss about creating one ore more companion repositories for the detailed tests and tutorials.
For the moment, with Benjamin we made, at least, these tests optional at ecbuild time, especially because they rely on heavy test datasets which are not distributed with Vind. One more reason to put them apart.

[jeromebarre]

Yes this what I have in mind for our NASA applications with VIND. To have separate repos that do the heavier and application specific tier of testing.

[benjaminmenetrier]

Thank you for this discussion thread. I agree that big tests could be moved to downstream repositories. But having worked on SABER for a long time, I also know from experience that downstream repositories are a pain when you want to merge contributions that affect the tests results.

So my feelings are mixed about doing this now. I think the situation is still manageable since we have tests sets split into separate directories, that they can be executed or not depending on build arguments.