You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+17-62Lines changed: 17 additions & 62 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,31 +22,7 @@ It consists of the following submodules:
22
22
### Publishers
23
23
24
24
`message-queue-toolkit` provides base classes for implementing publishers for each of the supported protocol.
25
-
26
-
#### Mono-schema publishers
27
-
28
-
Mono-schema publishers only support a single message type and are simpler to implement. They expose the following public methods:
29
-
30
-
*`constructor()`, which accepts the following parameters:
31
-
*`dependencies` – a set of dependencies depending on the protocol;
32
-
*`options`, composed by
33
-
*`messageSchema` – the `zod` schema for the message;
34
-
*`messageTypeField` - which field in the message describes the type of a message. This field needs to be defined as `z.literal` in the schema;
35
-
*`locatorConfig` - configuration for resolving existing queue and/or topic. Should not be specified together with the `creationConfig`.
36
-
*`creationConfig` - configuration for queue and/or topic to create, if one does not exist. Should not be specified together with the `locatorConfig`.
37
-
*`init()`, prepare publisher for use (e. g. establish all necessary connections), it will be called automatically by `publish()` if not called before explicitly (lazy loading).
38
-
*`close()`, stop publisher use (e. g. disconnect);
39
-
*`publish()`, send a message to a queue or topic. It accepts the following parameters:
40
-
*`message` – a message following a `zod` schema;
41
-
*`options` – a protocol-dependent set of message parameters. For more information please check documentation for options for each protocol: [AMQP](https://amqp-node.github.io/amqplib/channel_api.html#channel_sendToQueue), [SQS](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sqs/interfaces/sendmessagecommandinput.html) and [SNS](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sns/interfaces/publishcommandinput.html).
42
-
43
-
> **_NOTE:_** See [SqsPermissionPublisherMonoSchema.ts](./packages/sqs/test/publishers/SqsPermissionPublisherMonoSchema.ts) for a practical example.
44
-
45
-
> **_NOTE:_** Lazy loading is not supported for AMQP publishers.
46
-
47
-
#### Multi-schema publishers
48
-
49
-
Multi-schema publishers support multiple messages types. They implement the following public methods:
25
+
They implement the following public methods:
50
26
51
27
*`constructor()`, which accepts the following parameters:
52
28
*`dependencies` – a set of dependencies depending on the protocol;
@@ -61,35 +37,14 @@ Multi-schema publishers support multiple messages types. They implement the foll
61
37
*`message` – a message following one of the `zod` schemas, supported by the publisher;
62
38
*`options` – a protocol-dependent set of message parameters. For more information please check documentation for options for each protocol: [AMQP](https://amqp-node.github.io/amqplib/channel_api.html#channel_sendToQueue), [SQS](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sqs/interfaces/sendmessagecommandinput.html) and [SNS](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-sns/interfaces/publishcommandinput.html).
63
39
40
+
> **_NOTE:_** See [SqsPermissionPublisher.ts](./packages/sqs/test/publishers/SqsPermissionPublisher.ts) for a practical example.
41
+
> **_NOTE:_** Lazy loading is not supported for AMQP publishers.
42
+
64
43
65
44
### Consumers
66
45
67
46
`message-queue-toolkit` provides base classes for implementing consumers for each of the supported protocol.
68
-
69
-
#### Mono-schema consumers
70
-
71
-
Mono-schema consumers only support a single message type and are simpler to implement. They expose the following public methods:
72
-
73
-
*`constructor()`, which accepts the following parameters:
74
-
*`dependencies` – a set of dependencies depending on the protocol;
75
-
*`options`, composed by
76
-
*`messageSchema` – the `zod` schema for the message;
77
-
*`messageTypeField` - which field in the message is used for resolving the message type (for observability purposes);
78
-
*`queueName`; (for SNS publishers this is a misnomer which actually refers to a topic name)
79
-
*`locatorConfig` - configuration for resolving existing queue and/or topic. Should not be specified together with the `creationConfig`.
80
-
*`creationConfig` - configuration for queue and/or topic to create, if one does not exist. Should not be specified together with the `locatorConfig`.
81
-
*`subscriptionConfig` - SNS SQS consumer only - configuration for SNS -> SQS subscription to create, if one doesn't exist.
82
-
*`consumerOverrides` – available only for SQS consumers;
83
-
*`subscribedToTopic` – parameters for a topic to use during creation if it does not exist. Ignored if `queueLocator.subscriptionArn` is set. Available only for SNS consumers;
84
-
*`init()`, prepare consumer for use (e. g. establish all necessary connections);
85
-
*`close()`, stop listening for messages and disconnect;
86
-
*`processMessage()`, which accepts as parameter a `message` following a `zod` schema and should be overridden with logic on what to do with the message;
87
-
*`start()`, which invokes `init()` and `processMessage()` and handles errors.
88
-
*`preHandlerBarrier`, which accepts as a parameter a `message` following a `zod` schema and can be overridden to enable the barrier pattern (see [Barrier pattern](#barrier-pattern))
89
-
90
-
> **_NOTE:_** See [SqsPermissionConsumerMonoSchema.ts](./packages/sqs/test/consumers/SqsPermissionConsumerMonoSchema.ts) for a practical example.
91
-
92
-
#### Multi-schema consumers
47
+
They expose the following public methods:
93
48
94
49
Multi-schema consumers support multiple message types via handler configs. They expose the following public methods:
95
50
@@ -106,17 +61,18 @@ Multi-schema consumers support multiple message types via handler configs. They
106
61
*`subscribedToTopic` – parameters for a topic to use during creation if it does not exist. Ignored if `queueLocator.subscriptionArn` is set. Available only for SNS consumers;
107
62
*`init()`, prepare consumer for use (e. g. establish all necessary connections);
108
63
*`close()`, stop listening for messages and disconnect;
64
+
*`start()`, which invokes `init()`.
109
65
110
-
*`processMessage()`, which accepts as parameter a `message` following a `zod` schema and should be overridden with logic on what to do with the message;
111
-
*`start()`, which invokes `init()` and `processMessage()` and handles errors.
66
+
> **_NOTE:_** See [SqsPermissionConsumer.ts](./packages/sqs/test/consumers/SqsPermissionConsumer.ts) for a practical example.
112
67
113
-
##### Multi-schema handler definition
68
+
69
+
##### How to define a handler
114
70
115
71
You can define handlers for each of the supported messages in a type-safe way using the MessageHandlerConfigBuilder.
@@ -176,7 +132,7 @@ export class TestConsumerMultiSchema extends AbstractSqsConsumerMultiSchema<
176
132
177
133
#### Error Handling
178
134
179
-
When implementing message handler in consumer (by overriding the `processMessage()` method), you are expected to return an instance of `Either`, containing either an error `retryLater`, or result `success`. In case of `retryLater`, the abstract consumer is instructed to requeue the message. Otherwise, in case of success, the message is finally removed from the queue. If an error is thrown while processing the message, the abstract consumer will also requeue the message. When overriding the `processMessage()` method, you should leverage the possible types to process the message as you need.
135
+
When implementing a handler, you are expected to return an instance of `Either`, containing either an error `retryLater`, or result `success`. In case of `retryLater`, the abstract consumer is instructed to requeue the message. Otherwise, in case of success, the message is finally removed from the queue. If an error is thrown while processing the message, the abstract consumer will also requeue the message.
180
136
181
137
#### Schema Validation and Deserialization
182
138
@@ -189,23 +145,22 @@ If
189
145
190
146
Then the message is automatically nacked without requeueing by the abstract consumer and processing fails.
191
147
192
-
> **_NOTE:_** See [userConsumerSchemas.ts](./packages/sqs/test/consumers/userConsumerSchemas.ts) and [SqsPermissionsConsumerMonoSchema.spec.ts](./packages/sqs/test/consumers/SqsPermissionsConsumerMonoSchema.spec.ts) for a practical example.
148
+
> **_NOTE:_** See [userConsumerSchemas.ts](./packages/sqs/test/consumers/userConsumerSchemas.ts) and [SqsPermissionsConsumer.spec.ts](./packages/sqs/test/consumers/SqsPermissionsConsumer.spec.ts) for a practical example.
193
149
194
150
### Barrier pattern
195
151
The barrier pattern facilitates the out-of-order message handling by retrying the message later if the system is not yet in the proper state to be able to process that message (e. g. some prerequisite messages have not yet arrived).
196
152
197
-
To enable this pattern you should implement`preHandlerBarrier` in order to define the conditions for starting to process the message.
153
+
To enable this pattern you should define`preHandlerBarrier` on your message handler in order to define the conditions for starting to process the message.
198
154
If the barrier method returns `false`, message will be returned into the queue for the later processing. If the barrier method returns `true`, message will be processed.
199
155
200
-
> **_NOTE:_** See [SqsPermissionConsumerMonoSchema.ts](./packages/sns/test/consumers/SnsSqsPermissionConsumerMonoSchema.ts) for a practical example on mono consumers.
201
-
> **_NOTE:_** See [SqsPermissionConsumerMultiSchema.ts](./packages/sns/test/consumers/SnsSqsPermissionConsumerMultiSchema.ts) for a practical example on multi consumers.
156
+
> **_NOTE:_** See [SqsPermissionConsumer.ts](./packages/sns/test/consumers/SnsSqsPermissionConsumer.ts) for a practical example.
202
157
203
158
204
159
## Fan-out to Multiple Consumers
205
160
206
161
SQS queues are built in a way that every message is only consumed once, and then deleted. If you want to do fan-out to multiple consumers, you need SNS topic in the middle, which is then propagated to all the SQS queues that have subscribed.
207
162
208
-
> **_NOTE:_** See [SnsPermissionPublisher.ts](./packages/sns/test/publishers/SnsPermissionPublisherMonoSchema.ts) and [SnsSqsPermissionConsumerMonoSchema.ts](./packages/sns/test/consumers/SnsSqsPermissionConsumerMonoSchema.ts) for a practical example.
163
+
> **_NOTE:_** See [SnsPermissionPublisher.ts](./packages/sns/test/publishers/SnsPermissionPublisher.ts) and [SnsSqsPermissionConsumerMonoSchema.ts](./packages/sns/test/consumers/SnsSqsPermissionConsumer.ts) for a practical example.
209
164
210
165
## Automatic Queue and Topic Creation
211
166
@@ -220,7 +175,7 @@ In certain cases you want to await until certain publisher publishes a message,
220
175
In order to enable this functionality, configure spyHandler on the publisher or consumer:
0 commit comments