Skip to content

Add job to launcher routing #176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 41 additions & 16 deletions docs/docs/bridges/symfony-framework.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ you will be able to register these using configuration:

# config/packages/yokai_batch.yaml
yokai_batch:
launcher:
default: simple
launchers:
simple: ...
async: ...
launcher:
default: simple
launchers:
simple: ...
async: ...

.. note::
If you do not configure anything here, you will be using the
Expand All @@ -46,6 +46,31 @@ All ``launchers`` are configured using a DSN, every scheme has it’s own associ

* ``service``: the id of the service to use (required, an exception will be thrown otherwise)


| You might define multiple job launchers, and will want to configure the relation between job and launcher.
| For instance, you might prefer running some jobs with an async job launcher, but not all.
| You can configure this routing like the following:

.. code-block:: yaml

# config/packages/yokai_batch.yaml
yokai_batch:
launcher:
default: simple
launchers:
simple: simple://simple
console: console://console
routing:
export_job_name: simple
import_job_name: console

.. note::
It is not required to configure every single job in the ``routing``.
The ``default`` will be the fallback for all jobs you did not not configured in the ``routing``.

.. note::
If you configure ``config.launcher.routing``, it will replace your configured default from autowiring perspective.

.. seealso::
| :doc:`What is a job launcher? </core-concepts/job-launcher>`

Expand All @@ -64,12 +89,12 @@ You can have only one storage for your ``JobExecution``, and you have several op

# config/packages/yokai_batch.yaml
yokai_batch:
storage:
filesystem: ~
# Or with yokai/batch-doctrine-dbal (& doctrine/dbal)
# dbal: ~
# Or with a service of yours
# service: ~
storage:
filesystem: ~
# Or with yokai/batch-doctrine-dbal (& doctrine/dbal)
# dbal: ~
# Or with a service of yours
# service: ~

.. note::
| The default storage is ``filesystem``, because it only requires a writeable filesystem.
Expand Down Expand Up @@ -98,11 +123,11 @@ You can configure what your id will be like:

# config/packages/yokai_batch.yaml
yokai_batch:
id: uniqid
# Or with yokai/batch-symfony-uid (& symfony/uid)
# id: symfony.uuid.random
# id: symfony.uuid.time
# id: symfony.ulid
id: uniqid
# Or with yokai/batch-symfony-uid (& symfony/uid)
# id: symfony.uuid.random
# id: symfony.uuid.time
# id: symfony.ulid

User interface to visualize ``JobExecution``
------------------------------------------------------------
Expand Down
10 changes: 5 additions & 5 deletions docs/docs/bridges/symfony-messenger.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,11 @@ and will not want to run all jobs on the same transport.

# config/packages/yokai_batch.yaml
yokai_batch:
launchers:
messenger:
routing:
export_job_name: async_with_low_priority
import_job_name: async_with_high_priority
launchers:
messenger:
routing:
export_job_name: async_with_low_priority
import_job_name: async_with_high_priority

.. seealso::
| :doc:`What is a job launcher? </core-concepts/job-launcher>`
Expand Down
3 changes: 3 additions & 0 deletions docs/docs/core-concepts/job-launcher.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ What types of launcher exists?
* `SimpleJobLauncher <https://github.com/yokai-php/batch/tree/0.x/src/Launcher/SimpleJobLauncher.php>`__:
execute the job directly in the same PHP process.

* `RoutingJobLauncher <https://github.com/yokai-php/batch/tree/0.x/src/Launcher/RoutingJobLauncher.php>`__:
pick the appropriate job launcher for each job, based on the configuration you provide.

**Launchers from bridges:**

* From ``symfony/console`` bridge:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
* @phpstan-type LauncherConfig array{
* default: string|null,
* launchers: array<string, string>,
* routing?: array<string, string>,
* messenger?: array{
* routing: array<string, string>,
* },
Expand Down Expand Up @@ -138,6 +139,11 @@ private function launcher(): ArrayNodeDefinition
->end()
->end()
->end()
->arrayNode('routing')
->normalizeKeys(false)
->useAttributeAsKey('name')
->scalarPrototype()->end()
->end()
->arrayNode('messenger')
->children()
->arrayNode('routing')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

namespace Yokai\Batch\Bridge\Symfony\Framework\DependencyInjection;

use Symfony\Component\DependencyInjection\Compiler\ServiceLocatorTagPass;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Definition;
use Symfony\Component\DependencyInjection\Exception\LogicException;
use Symfony\Component\DependencyInjection\Reference;
Expand All @@ -13,6 +15,7 @@
use Yokai\Batch\Bridge\Symfony\Messenger\DispatchMessageJobLauncher;
use Yokai\Batch\Bridge\Symfony\Messenger\MessengerJobsConfiguration;
use Yokai\Batch\Launcher\JobLauncherInterface;
use Yokai\Batch\Launcher\RoutingJobLauncher;
use Yokai\Batch\Launcher\SimpleJobLauncher;
use Yokai\Batch\Storage\JobExecutionStorageInterface;

Expand Down Expand Up @@ -40,6 +43,25 @@ public static function fromDsn(string $dsn): Definition|Reference
};
}

/**
* Create the {@see RoutingJobLauncher} service definition when has configuration for it.
*
* @param array<string, string> $launchers
* @param array<string, string> $routing
*/
public static function routing(
ContainerBuilder $container,
array $launchers,
string $default,
array $routing,
): Definition {
return new Definition(RoutingJobLauncher::class, [
'$launchers' => ServiceLocatorTagPass::register($container, $launchers),
'$default' => new Reference($default),
'$routing' => $routing,
]);
}

private static function simple(): Definition
{
return new Definition(SimpleJobLauncher::class, [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ private function configureLauncher(ContainerBuilder $container, array $config):

$container->setParameter('yokai_batch.launcher.messenger_routing', $config['messenger']['routing'] ?? []);

$launcherIdPerLauncherName = [];
$idPerLauncherName = [];
foreach ($config['launchers'] as $name => $dsn) {
$definitionOrReference = JobLauncherDefinitionFactory::fromDsn($dsn);
if ($definitionOrReference instanceof Definition) {
Expand All @@ -156,15 +156,23 @@ private function configureLauncher(ContainerBuilder $container, array $config):
$launcherId = (string)$definitionOrReference;
}

$launcherIdPerLauncherName[$name] = $launcherId;
$idPerLauncherName[$name] = $launcherId;
$parameterName = $name . 'JobLauncher';
$container->registerAliasForArgument($launcherId, JobLauncherInterface::class, $parameterName);
}

$container->setAlias(
JobLauncherInterface::class,
$launcherIdPerLauncherName[$config['default']],
);
$default = $idPerLauncherName[$config['default']];

$routing = $config['routing'] ?? [];
if ($routing !== []) {
$container->setDefinition(
$launcherId = 'yokai_batch.job_launcher.routing',
JobLauncherDefinitionFactory::routing($container, $idPerLauncherName, $default, $routing),
);
$default = $launcherId;
}

$container->setAlias(JobLauncherInterface::class, $default);
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
use Yokai\Batch\Factory\JobExecutionParametersBuilderInterface;
use Yokai\Batch\Factory\UniqidJobExecutionIdGenerator;
use Yokai\Batch\Launcher\JobLauncherInterface;
use Yokai\Batch\Launcher\RoutingJobLauncher;
use Yokai\Batch\Launcher\SimpleJobLauncher;
use Yokai\Batch\Storage\FilesystemJobExecutionStorage;
use Yokai\Batch\Storage\JobExecutionStorageInterface;
Expand Down Expand Up @@ -77,14 +78,23 @@ public function storage(): \Generator
/**
* @dataProvider launcher
*/
public function testLauncher(array $config, \Closure|null $configure, array $launchers, string $default): void
{
public function testLauncher(
array $config,
\Closure|null $configure,
array $launchers,
string $default,
\Closure|null $assert = null,
): void {
$container = $this->createContainer($config, $configure);

self::assertSame($default, (string)$container->getAlias(JobLauncherInterface::class));
foreach ($launchers as $id => $class) {
self::assertSame($class, $container->getDefinition($id)->getClass());
}

if ($assert) {
$assert($container);
}
}

public function launcher(): \Generator
Expand Down Expand Up @@ -112,6 +122,43 @@ public function launcher(): \Generator
],
'yokai_batch.job_launcher.messenger',
];
yield 'Messenger launcher routing' => [
[
'launcher' => [
'default' => 'messenger',
'launchers' => [
'messenger' => 'messenger://messenger',
],
'messenger' => [
'routing' => [
'job1' => 'async',
'job2' => 'sync',
],
],
],
],
null,
[
'yokai_batch.job_launcher.messenger' => DispatchMessageJobLauncher::class,
],
'yokai_batch.job_launcher.messenger',
function (ContainerBuilder $container) {
$messengerJobsConfiguration = $container->getDefinition('yokai_batch.job_launcher.messenger')
->getArgument('$messengerJobsConfiguration');
self::assertInstanceOf(Definition::class, $messengerJobsConfiguration);
self::assertSame(
'%yokai_batch.launcher.messenger_routing%',
$messengerJobsConfiguration->getArgument('$routing'),
);
self::assertSame(
[
'job1' => 'async',
'job2' => 'sync',
],
$container->getParameter('yokai_batch.launcher.messenger_routing'),
);
},
];
yield 'Service launcher' => [
[
'launcher' => [
Expand All @@ -128,6 +175,39 @@ public function launcher(): \Generator
['app.job_launcher' => BufferingJobLauncher::class],
'app.job_launcher',
];
yield 'Routing launcher' => [
[
'launcher' => [
'default' => 'simple',
'launchers' => [
'simple' => 'simple://simple',
'messenger' => 'messenger://messenger',
'console' => 'console://console',
],
'routing' => [
'job1' => 'messenger',
'job2' => 'console',
],
],
],
null,
[
'yokai_batch.job_launcher.simple' => SimpleJobLauncher::class,
'yokai_batch.job_launcher.messenger' => DispatchMessageJobLauncher::class,
'yokai_batch.job_launcher.console' => RunCommandJobLauncher::class,
'yokai_batch.job_launcher.routing' => RoutingJobLauncher::class,
],
'yokai_batch.job_launcher.routing',
function (ContainerBuilder $container) {
self::assertSame(
[
'job1' => 'messenger',
'job2' => 'console',
],
$container->getDefinition('yokai_batch.job_launcher.routing')->getArgument('$routing'),
);
},
];
}

/**
Expand Down
46 changes: 46 additions & 0 deletions src/batch/src/Launcher/RoutingJobLauncher.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<?php

declare(strict_types=1);

namespace Yokai\Batch\Launcher;

use Psr\Container\ContainerInterface;
use Yokai\Batch\Exception\UnexpectedValueException;
use Yokai\Batch\JobExecution;

/**
* This {@see JobLauncherInterface} is a proxy locator to other {@see JobLauncherInterface}.
* It will allow you to configure the relation between job and launcher.
* It requires a collection of {@see JobLauncherInterface} along with a default one, and some configuration.
*
* Whenever it is asked to launch a job, if that job is configured with a routing entry,
* the corresponding {@see JobLauncherInterface} will be used.
* Otherwise, this is the provided default {@see JobLauncherInterface} that will be used.
*/
final class RoutingJobLauncher implements JobLauncherInterface
{
public function __construct(
private ContainerInterface $launchers,
private JobLauncherInterface $default,
/**
* @var array<string, string>
*/
private array $routing,
) {
}

public function launch(string $name, array $configuration = []): JobExecution
{
$launcherName = $this->routing[$name] ?? null;
if ($launcherName === null) {
$launcher = $this->default;
} else {
$launcher = $this->launchers->get($launcherName);
if (!$launcher instanceof JobLauncherInterface) {
throw UnexpectedValueException::type(JobLauncherInterface::class, $launcher);
}
}

return $launcher->launch($name, $configuration);
}
}
Loading
Loading