[Host.Outbox] Is it possible create outbox table with migrations?

[poly-sladkiy]

The problem is that in tests the database is created on the fly, and OutboxSendingTask tries to create a table and the test fails on first Publish in SlimMessageBus.Host.MessageBusBase.ProducePublish

Exception:

Details

SlimMessageBus.MessageBusException: Outbox schema migration failed
 ---> Npgsql.PostgresException (0x80004005): 3D000: database "DCS_FT_6f-a98a-1a4352cfcbeb" does not exist
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore|209_0(NpgsqlConnector conn, String username, SslMode sslMode, GssEncryptionMode gssEncMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.PoolingDataSource.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.PoolingDataSource.<Get>g__RentAsync|33_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlConnection.<Open>g__OpenAsync|42_0(Boolean async, CancellationToken cancellationToken)
   at SlimMessageBus.Host.Outbox.PostgreSql.Repositories.PostgreSqlOutboxMessageRepository.EnsureConnection(CancellationToken cancellationToken)
   at SlimMessageBus.Host.Outbox.PostgreSql.Services.PostgreSqlOutboxMigrationService.EnsureMigrationHistoryTable(CancellationToken cancellationToken)
   at SlimMessageBus.Host.Outbox.PostgreSql.Services.PostgreSqlOutboxMigrationService.Migrate(CancellationToken token)
   at SlimMessageBus.Host.Outbox.Services.OutboxSendingTask`1.MigrateSchema(IServiceProvider serviceProvider, CancellationToken cancellationToken)
  Exception data:
    Severity: FATAL
    SqlState: 3D000
    MessageText: database "DCS_FT_6f-a98a-1a4352cfcbeb" does not exist
    File: postinit.c
    Line: 948
    Routine: InitPostgres
   --- End of inner exception stack trace ---
   at SlimMessageBus.Host.Outbox.Services.OutboxSendingTask`1.MigrateSchema(IServiceProvider serviceProvider, CancellationToken cancellationToken)
   at SlimMessageBus.Host.Outbox.Services.OutboxSendingTask`1.MigrateSchema(IServiceProvider serviceProvider, CancellationToken cancellationToken)
   at SlimMessageBus.Host.MessageBusBase.OnBusLifecycle(MessageBusLifecycleEventType eventType)
   at SlimMessageBus.Host.MessageBusBase.OnBusLifecycle(MessageBusLifecycleEventType eventType)
   at SlimMessageBus.Host.Collections.AsyncTaskList.EnsureAllFinished()

[zarusz]

Hey, please elaborate and help me understand the ask here.

The Outbox plugins manage migrations outside of EF Migration but push keep track of what's already migrated in a similar table that EF.
The goal here was to not depend on the EF directly but just follow its migration table structure. The user can also change the name of the migraiton table to decouple EF migrations from SMB migrations.

The exception that you reported seems to be an issue with connectivity to the PG database.

[poly-sladkiy]

Hi!

The goal here was to not depend on the EF directly but just follow its migration table structure

Okey, I get it. Good solution!

In my case the DB created lately than SMB try to create an outbox table. Therefore, when SMB tries to connect to the database, it crashes.

This happens because the OutboxSendingTask service implements the IMessageBusLifecycleInterceptor interface, which is called during the creation of the MessageBusBase. For my case it is KafkaMessageBus.

I have some ideas:

• Add a check until PostgreSqlOutboxMessageRepository.EnsureConnection connects to the database. (require library revision)
• Force OnBuildProvider or IMessageBusLifecycleInterceptor implementation to be called until the database is created.

I can try to create my own IMessageBusLifecycleInterceptor that will wait until the database is created.
Or do you see other solutions?

Forgot to write the stack: .net 10, asp.net core

SlimMessageBus libraries

    <PackageVersion Include="SlimMessageBus" Version="3.0.0" />
    <PackageVersion Include="SlimMessageBus.Host" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Memory" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Kafka" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Serialization.Json" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Outbox" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Outbox.PostgreSql.DbContext" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Serialization.Hybrid" Version="3.3.6" />
    <PackageVersion Include="SlimMessageBus.Host.Serialization.GoogleProtobuf" Version="3.3.6" />

[poly-sladkiy]

I found solutions for this case:

1. Create database before Host starts
2. Kludge, create custom IMessageBusLifecycleInterceptor with CancellationTokenSource, and register it before SMB

I would recommend adding Retry to connect and making the error more obvious during startup rather than when first calling SMB methods.

[zarusz]

@poly-sladkiy thanks for sharing this. If you like to propose and work on an enhancement here I will be glad to accept the PR.