Skip to content

Conversation

@dennisvang
Copy link
Contributor

@dennisvang dennisvang commented Oct 31, 2025

This PR enables the configuration of multiple fixtures directories as follows:

bootstrap:
  db-fixtures-dirs:
    - "fixtures"
    - "more-fixtures"

or, using environment variables:

BOOTSTRAP_DBFIXTURESDIRS_0_=fixtures
BOOTSTRAP_DBFIXTURESDIRS_1_=more-fixtures

or, using command line arguments:

... -Dbootstrap.db-fixtures-dirs[0]=fixtures  -Dbootstrap.db-fixtures-dirs[1]=more-fixtures

All JSON fixture files are added to a single, sorted, resources list.

This gives more flexibility in case e.g. users want to load additional fixture files from container secrets, or similar scenarios.

TODO

  • fix tests

@dennisvang
Copy link
Contributor Author

dennisvang commented Nov 11, 2025

Tests are refusing to run in the workflow on windows runner, apparently due to some config issue:

...
Error:  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 12.82 s <<< FAILURE! -- in org.fairdatapoint.acceptance.actuator.List_Info_GET
Error:  org.fairdatapoint.acceptance.actuator.List_Info_GET.res200 -- Time elapsed: 0.016 s <<< ERROR!
java.lang.IllegalStateException: Failed to load ApplicationContext for [WebMergedContextConfiguration@c382f4f testClass = org.fairdatapoint.acceptance.actuator.List_Info_GET ...
...

However, the exact same tests do run on the ubuntu runner, although some of them fail:

...
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s -- in org.fairdatapoint.entity.schema.SemVerTest
[INFO] Running org.fairdatapoint.database.db.repository.bootstrap.FixtureHistoryRepositoryTests
Error:  Tests run: 4, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 4.852 s <<< FAILURE! -- in org.fairdatapoint.database.db.repository.bootstrap.FixtureHistoryRepositoryTests
Error:  org.fairdatapoint.database.db.repository.bootstrap.FixtureHistoryRepositoryTests.testSave -- Time elapsed: 0.019 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <1> but was: <18>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:161)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:632)
	at org.fairdatapoint.database.db.repository.bootstrap.FixtureHistoryRepositoryTests.testSave(FixtureHistoryRepositoryTests.java:51)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
...

The latter is due to the change in default value from bootstrap.enabled=false to bootstrap.enabled=true.

@dennisvang
Copy link
Contributor Author

dennisvang commented Nov 11, 2025

So now all tests pass on ubuntu and mac, but still failing to load application context on the windows runner.

Probably an issue with the workflow itself, but this has not changed, so perhaps something to do with workflow dependencies?

Update:

The windows runner logs show it is still using ikalnytskyi/action-setup-postgres@v7, even though that was upgraded to v8 in FAIRDataTeam/github-workflows#29 (v2.4.1). The problem here was that the major tag v2 for github-workflows had not been updated to point to v2.4.1.

However, even after fixing this in github-workflows and forcing a clean re-run of the workflow (amend commit message and force push), the windows runner still fails. Confirmed it is now using action-setup-postgres v8.

The test assertions expect the db to be empty.
Previously, bootstrap.enabled was false by default, but now it is true by default.
Therefore we need to disable bootstrap explicitly in these tests.
@dennisvang dennisvang force-pushed the proposal/634-multiple-dirs branch from c1ee079 to a79f9c8 Compare November 11, 2025 13:26
@dennisvang
Copy link
Contributor Author

Still no success:

image

Postgres connection apears OK (check added in github-workflows v2.4.2).

Could this be a path issue (windows vs unix paths)?

@dennisvang
Copy link
Contributor Author

dennisvang commented Nov 12, 2025

So, if we disable bootstrapping altogether, all tests pass, including the windows tests.

The data are still coming from the test sql migrations, which are still active... (but, then again, same holds for the non-windows runners)

@dennisvang dennisvang changed the title Allow loading of relational db fixtures from multiple directories Enable loading of relational db fixtures from multiple directories Nov 13, 2025
actually mongo is not used anymore, but spring will try to configure it because it is used in the spring-rdf-migration dependency
@dennisvang
Copy link
Contributor Author

dennisvang commented Nov 13, 2025

With proper logging enabled during tests, we now have the problem from the windows runner:

...
2025-11-13 10:43:33,149 42541 [main] WARN  org.flywaydb.core.internal.database.base.Database - Flyway upgrade recommended: PostgreSQL 18.0 is newer than this version of Flyway and support has not been tested. The latest supported version of PostgreSQL is 17.
2025-11-13 10:43:33,315 42707 [main] WARN  org.flywaydb.core.internal.database.base.Database - Flyway upgrade recommended: PostgreSQL 18.0 is newer than this version of Flyway and support has not been tested. The latest supported version of PostgreSQL is 17.
2025-11-13 10:43:33,719 43111 [main] WARN  org.hibernate.orm.deprecation - HHH90000025: PostgreSQLDialect does not need to be specified explicitly using 'hibernate.dialect' (remove the property setting and it will be selected by default)
2025-11-13 10:43:34,839 44231 [main] WARN  org.springframework.web.context.support.GenericWebApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'repositoryPopulator' defined in class path resource [org/fairdatapoint/config/BootstrapConfig.class]: 
  Failed to instantiate [org.springframework.data.repository.init.Jackson2RepositoryPopulatorFactoryBean]: 
    Factory method 'repositoryPopulator' threw exception with message: 
      Illegal char <*> at index 0: *.json
...

We can reproduce this type of error on unix by including a null character in the filename, which is illegal on unix. For example:

Path.of(fixturesDir).resolve("\0")

On unix we can simply do Path.resolve(*.json), but on windows that fails with 'Illegal char <*> at index 0: *.json', so we do use Path to handle slashes, but only add the * after converting back to string
This is needed because resolve fails on windows due to invalid path character *.
Note that we can safely concatenate /*.json (with the initial slash) because Path.of() removes any trailing slashes.
@dennisvang dennisvang force-pushed the proposal/634-multiple-dirs branch from 2e40309 to 81fb68c Compare November 13, 2025 16:18
@dennisvang dennisvang merged commit ce5b29b into feature/634-boostrapping-fdp Nov 13, 2025
13 checks passed
@dennisvang dennisvang deleted the proposal/634-multiple-dirs branch November 13, 2025 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants