TASK-10736: Documentation of new features and changelog of Momentum 5.1 (#800)

Signed-off-by: Doug Koerich <[email protected]>
Co-authored-by: Julie Zhao <[email protected]>
Co-authored-by: Kris Kelley <[email protected]>

Author: 3 people

content/momentum/4/eol-policy.md

@@ -63,7 +63,8 @@ Momentum version 5 became GA on March 1, 2025. Therefore:
 | Momentum 4.6.0                  | 2023/10/20 | 2024/12/19         | 2024/12/31³    |
 | Momentum 4.7.0                  | 2023/12/19 | 2025/10/17         | 2027/3/1       |
 | Momentum 4.8.0                  | 2024/10/17 | 2026/3/1           | 2027/3/1       |
-| **Momentum 5.0.0**              | 2025/3/1   | TBD                | TBD            |
+| Momentum 5.0.0                  | 2025/3/1   | 2026/7/1           | TBD            |
+| **Momentum 5.1.0**              | 2025/7/1   | TBD                | TBD            |
 
 > ¹ Momentum 4.4.x was superseded by 4.6, which was the last version supporting CentOS 7.
 >

content/momentum/4/hooks/core-smtp-tls-reporting.md

@@ -0,0 +1,123 @@
+---
+lastUpdated: "05/10/2025"
+title: "smtp_tls_reporting"
+description: "hook invoked after every TLS events for reporting purpose rfc8460 TLSRPT"
+---
+
+<a name="hooks.core.smtp_tls_reporting"></a>
+## Name
+
+smtp_tls_reporting - This hook is added in 5.1 and allows you inspect a SMTP TLS reporting data
+ point in JSON format.
+
+## Synopsis
+
+`#include "hooks/core/smtp_tls_reporting.h"`
+
+`int core_smtp_tls_reporting(void closure, struct json_object *json)`
+
+
+## Description
+
+This hook is called upon:
+- any TLSRPT (rfc8460) defined failures, before a TLS connection is attempted,
+  normally during TLS policy (including MTA-STS, TLSA/DANE) fetching stage.
+- TLS negotiation failures or successes during outbound delivery when MTA-STS or TLSA/DANE is enabled.
+  **Currently, only enabled on domains with successfully fetched MTA-STS policies or DANE TLSA records **.
+
+The JSON fields and values are defined in `tlsrpt.h`, with most of the field names the same as
+ defined in the RFC: https://datatracker.ietf.org/doc/html/rfc8460.
+
+The following JSON fields are not defined in the RFC:
+*   `epoch` - epoch time of when the hook is invoked
+*   `type`  - whether the data is for a successful TLS connection or a failure.
+              `0` - failure; `1` - success
+
+**An example JSON for a success**:
+
+```
+{
+    "epoch": 1746712864,
+    "type": 1,
+    "policy": {
+        "policy-type": "sts",
+        "policy-domain": "test.bird.com",
+        "policy-string": [
+            "version: STSv1",
+            "mode: enforce",
+            "mx: mx.bird.com",
+            "mx: server.ectest.OMNITI.com",
+            "max_age: 604800"
+        ]
+    },
+    "sending-mta-ip": "127.0.0.1",
+    "receiving-mx-hostname": "server.ectest.OMNITI.com",
+    "receiving-ip": "127.0.0.1"
+}
+```
+
+**An example JSON for a failure**:
+
+```
+{
+    "epoch": 1746629177,
+    "type": 0,
+    "policy": {
+        "policy-type": "sts",
+        "policy-domain": "mismatch.cert.com",
+        "policy-string": [
+            "version: STSv1",
+        "mode: enforce",
+        "mx: test.bird.com",
+        "max_age: 86400"
+        ]
+    },
+    "result-type": "certificate-host-mismatch",
+    "failure-reason-code": "4.7.5 [internal] SSL certificate subject does not match host",
+    "sending-mta-ip": "127.0.0.1",
+    "receiving-mx-hostname": "test.BIRD.com",
+    "receiving-ip": "127.0.0.1"
+}
+```
+
+
+**Return Values**
+
+This hook returns `int`, but for now the return value has no significance.
+
+
+**Threading**
+
+This hook could be called in any thread. Please avoid doing time consuming tasks in the hook's
+ implementation.
+
+
+### Example: a Lua implementation of the hook to log the JSON into the `paniclog`
+
+```
+ require("msys.core");
+ require("json")
+
+ local mod = {}
+
+ function mod:core_smtp_tls_reporting(js)
+   print("tls report: ", js)	-- log the whole JSON
+   if js.type == 0 then -- failure
+     print(string.format("TLSRPT: %s@%s@%s", js.policy["policy-domain"],
+			              js.policy["policy-type"], js["result-type"]))
+   else -- success
+     print(string.format("TLSRPT: %s@%s@%s", js.policy["policy-domain"],
+		  	              js.policy["policy-type"], "OK"))
+   end
+ end
+
+ msys.registerModule("tlsrpt", mod);
+```
+
+**Example of the paniclog output from the above Lua hook**:
+```
+1746712864:scriptlet: tls report:   { "epoch": 1746712864, "type": 1, "policy": { "policy-type": "sts", "policy-domain": "test.bird.com", "policy-string": [ "version: STSv1", "mode: enforce", "mx: mx.bird.com", "mx: server.ectest.OMNITI.com", "max_age: 604800" ] }, "sending-mta-ip": "127.0.0.1", "receiving-mx-hostname": "server.ectest.OMNITI.com", "receiving-ip": "127.0.0.1" }
+1746712864:scriptlet: TLSRPT: test.bird.com@sts@OK
+1746719856:scriptlet: tls report:   { "epoch": 1746719856, "type": 0, "policy": { "policy-type": "sts", "policy-domain": "mismatch.cert.com", "policy-string": [ "version: STSv1", "mode: enforce", "mx: test.bird.com", "max_age: 86400" ] }, "result-type": "certificate-host-mismatch", "failure-reason-code": "4.7.5 [internal] SSL certificate subject does not match host", "sending-mta-ip": "127.0.0.1", "receiving-mx-hostname": "test.BIRD.com", "receiving-ip": "127.0.0.1" }
+1746719856:scriptlet: TLSRPT: mismatch.cert.com@sts@certificate-host-mismatch
+```

content/momentum/4/hooks/index.md

@@ -16,7 +16,8 @@ description: "This chapter includes hook point and C function reference material
 | [ec_httpsrv_register_auth](/momentum/4/apis-ec-httpsrv-register-auth)                 | Register an HTTP handler for authenticating a URI                             |
 | [ec_httpsrv_request_local_address](/momentum/4/apis-ec-httpsrv-request-local-address) | Returns the local IP address from the current session                         |
 | [ec_httpsrv_request_peer_address](/momentum/4/apis-ec-httpsrv-request-peer-address)   | Returns the remote peer address from the current session                      |
-| [inbound_smtp_tls_post_accept](/momentum/4/hooks/inbound-smtp-tls-post-accept)         | Modify the message state after the tls handshake in esmtp_tls_accept (available in 4.4.0 or higher)      |
+| [inbound_smtp_tls_post_accept](/momentum/4/hooks/inbound-smtp-tls-post-accept)        | Modify the message state after the tls handshake in esmtp_tls_accept (available in 4.4.0 or higher)      |
+| [core_smtp_tls_reporting](/momentum/4/hooks/core-smtp-tls-reporting)                  | Report TLS events for TLSRPT (TLS reporting)     |
 
 This chapter includes hook point and C function reference material that is specific to Momentum 4.
 

content/momentum/4/index.md

@@ -1,6 +1,6 @@
 ---
 lastUpdated: "09/30/2024"
-title: "Momentum 4.x"
+title: "Momentum 4.x and later"
 description: "Message Systems Inc Copyright 2014-2024 Message Systems Inc Confidential Proprietary Abstract This book documents Momentum 4 Document generated on 2024 Sep 30 Table of Contents Preface 1 Typographical Conventions Used in This Document I Introduction to Momentum 1 Components 2 Life of A Message 3 Roles and Behaviors 4 Licensed..."
 ---
 
@@ -14,7 +14,7 @@ Confidential & Proprietary.
 
 **Abstract**
 
-This book documents Momentum 4.
+This book documents Momentum 4 and later.
 
 Document generated on: 2024-Sep-30
 

content/momentum/4/lua/ref-msys-validate-openarc-sign.md

@@ -56,7 +56,7 @@ This function takes the following parameters:
 
     *   `header_canon` – header canonicalization setting.
 
-        Supported values are `relaxed`, `simple`. Defaults to `relaxed`.
+        The only supported value is `relaxed`.
 
     *   `body_canon` – body canonicalization setting
 

content/momentum/4/modules/auth-radius.md

@@ -16,15 +16,15 @@ The following example demonstrates how to configure Momentum to pass LOGIN crede
 ```
 # Configure the RADIUS client
 auth_radius {
-  NAS-IP-Address = 10.0.0.1 # the IP address of this SMTP server
+  NAS-IP-Address = fd01:345::1 # the IP address of this SMTP server
   server "one" {
     host = "radius-1.example.com"
     secret = "secret1"
     max_tries = "1"
     timeout = "30"
   }
-  server "two" {
-    host = "radius-2.example.com"
+  server "ipv6wport" {
+    host = "[2001:fd3::1]:2812"
     secret = "secret2"
     max_tries = "2"
     timeout = "30"
@@ -85,7 +85,7 @@ RADIUS servers can be defined using the dictionary syntax shown above; the dicti
 
 <dd>
 
-The hostname or IP address of the RADIUS server. If a colon is present in the string then the left side of the string will be used as the hostname/IP address and the right hand side will be used as the port number on the server. If left unspecified, the RADIUS standard port number of 1812 will be used.
+The hostname or IP address of the RADIUS server. The hostname/IP address can be followed by a colon and the port number on the server. If left unspecified, the RADIUS standard port number of 1812 will be used.  An IPv6 address must be enclosed in brackets if a port was added.
 
 </dd>
 

content/momentum/4/modules/index.md

@@ -1,5 +1,5 @@
 ---
-lastUpdated: "03/01/2025"
+lastUpdated: "05/30/2025"
 title: "Category File"
 type: "custom"
 name: "Modules Reference"
@@ -63,7 +63,7 @@ description: "Table of Contents 71 1 Introduction 71 2 ac auth Authentication Ha
 | [openarc](/momentum/4/modules/openarc) | Open Source ARC |
 | [opendkim](/momentum/4/modules/opendkim) | Open Source DKIM |
 | [outbound_audit](/momentum/4/modules/outbound-audit) | Outbound traffic analytics |
-| [outbound_smtp_auth(modules.outbound_smtp_auth.php) |
+| [outbound_smtp_auth](/momentum/4/modules/outbound-smtp-auth) | Outbound authentication |
 | [persist_io](/momentum/4/modules/persistio) | Persistent IO Wrapper |
 | [pipe_io](/momentum/4/modules/pipeio) | Pipe IO Wrapper |
 | [pipe_transport](/momentum/4/modules/pipe-transport) | Module |

content/momentum/4/modules/mail-loop.md

@@ -8,7 +8,7 @@ description: "The mail loop module provides automatic suppression of potential m
 
 The mail_loop module provides automatic suppression of potential mail loops with two standard mechanisms:
 
-*   Suppression of delivery attempts to any configured IP interfaces on the machine.
+*   Suppression of delivery attempts to any configured IP interfaces on the machine, including IPv6.
 
 *   Suppression of messages with more than a specified number of Received headers.
 

content/momentum/4/modules/mxip.md

@@ -63,7 +63,7 @@ mxip.example.		86400	IN	NS	localhost.
 
 The mxip module implements a dns_get_As hook in order to augment the DNS resolution behaviour. If your integration or deployment also implements a dns_get_As hook, then you may not be able to use the mxip module.
 
-The mxip module only supports IPv4 addresses in the hostname field of MX records. IPv6 addresses are explicitly not supported by the mxip module.
+The mxip module supports IPv4 (and IPv6 as of Momentum 5.1) addresses in the hostname field of MX records.
 
 The mxip module can be configured as follows:
 

content/momentum/4/modules/outbound-smtp-auth.md

@@ -1,84 +1,94 @@
 ---
-lastUpdated: "03/26/2020"
+lastUpdated: "05/30/2025"
 title: "outbound_smtp_auth"
-description: "This module enables users to specify authentication parameters for a given set of messages so that Momentum will authenticate against the peer server when it sends outbound mail It currently supports the AUTH LOGIN and AUTH PLAIN methods of authentication You can specify the parameters in configuration or in lua..."
+description: "This module enables users to specify authentication parameters for a given set of messages so that Momentum will authenticate against the peer server when it sends outbound mail It currently supports the AUTH LOGIN, AUTH PLAIN and AUTH XOAUTH2 methods of authentication You can specify the parameters in configuration or in lua..."
 ---
 
-<a name="idp22419360"></a> 
+<a name="modules.outbound_smtp_auth"></a>
 
-This module enables users to specify authentication parameters for a given set of messages so that Momentum will authenticate against the peer server when it sends outbound mail. It currently supports the 'AUTH LOGIN' and 'AUTH PLAIN' methods of authentication. You can specify the parameters in configuration or in lua, or use a combination of both.
+This module enables users to specify authentication parameters for a given set of messages so that
+Momentum will authenticate against the peer server when it sends outbound mail. It currently
+supports the `AUTH LOGIN`, `AUTH PLAIN` and `AUTH XOAUTH2` methods of authentication.
+ You can specify the parameters in configuration or in lua, or use a combination of both.
 
 ### Note
 
 This module makes heavy use of message contexts to facilitate authentication. If it is enabled, you risk having extra I/O unless `keep_message_dicts_in_memory` is on.
 
-**Configuration Change. ** This feature is available in Momentum 4.2 and later.
-
 ### <a name="modules.outbound_smtp_auth.configuration"></a> Configuration
 
-Configuration variables are listed below. These values can all be changed and overridden by setting context variables with the same name as the options in lua. All variables are valid in the binding group, binding, domain, and global scopes.
-
-<dl class="variablelist">
-
-<dt>outbound_smtp_auth_key</dt>
-
-<dd>
+> This module is refactored in Momentum 5.1, but this feature is available in
+ Momentum 4.2 and later. `AUTH XOAUTH2` support is added in 5.1.
 
-A unique key that can be used in lua to look up authorization details in a database. It enables you to easily trigger custom behavior based on a configuration scope. The default value is `false`.
+Configuration variables are listed below. These values can all be changed and overridden by setting
+ message context variables with the same name as the options in lua.
+ All variables are valid in the binding group, binding, domain, and global scopes.
 
-</dd>
+<dl class="variablelist">
 
 <dt>outbound_smtp_auth_pass</dt>
 
 <dd>
 
-The password that will be passed to the remote server. The default value is `false`.
+The password or auth token (e.g. for `AUTH XOAUTH2`) that will be passed to the remote server.
+ It has no default value.
 
 ### Note
 
-Setting the password in configuration will leave it as plaintext. To set the password more securely, dynamically retrieve it from a data store in lua and set it in the context variable that corresponds to this option.
+Setting the password in configuration will leave it as plaintext.
+ To set the password more securely, it's recommended to dynamically retrieve it from a data store
+ in lua and set it in the context variable that corresponds to this option.
 
 </dd>
 
 <dt>outbound_smtp_auth_type</dt>
 
 <dd>
 
-Determines what authentication protocol should be used. The only supported values are 'PLAIN' and 'LOGIN'. The default value is `false`.
+Determines what authentication protocol should be used. The only supported values are `PLAIN`,
+ `LOGIN` and `XOAUTH2`. It has no default value.
 
 </dd>
 
 <dt>outbound_smtp_auth_user</dt>
 
 <dd>
 
-The username that will be passed to the remote server. The default value is `false`.
+The username that will be passed to the remote server. It has no default value.
 
 </dd>
 
 </dl>
 
 ### <a name="modules.outbound_smtp_auth.usage"></a> Usage
 
+A hook `outbound_smtp_auth_config(msg)` is added by this module to allow per message auth settings.
+ The settings in `ec_message` context will override the configuration values.
+ This hook is called in delivery/scheduler thread before sending SMTP `AUTH` command.
+ Please avoid blocking or lengthy operations when implementing this hook.
+
 Basic examples of usage are provided below.
 
-The following example shows how you can extend the new hook and set the username and password in lua.
+The following example shows how you can extend the new hook and set the username and password in lua
+ for each message.
 
-<a name="modules.outbound_smtp_auth.example.set_username_pw"></a> 
+<a name="modules.outbound_smtp_auth.example.set_username_pw"></a>
 
 
 ```
-function mod:outbound_smtp_auth_config(msg, ac, vctx)
-  print('NOTICE: outbound_smtp_auth_config Lua hook called');
-  print('NOTICE: msg:['.. tostring(msg) ..']')
-  msg:context_set(VCTX_MESS, 'outbound_smtp_auth_user', 'foo')
-  msg:context_set(VCTX_MESS, 'outbound_smtp_auth_pass', 'bar')
+function mod:outbound_smtp_auth_config(msg)
+  --print('NOTICE: outbound_smtp_auth_config Lua hook called');
+  msg:context_set(VCTX_MESS, 'outbound_smtp_auth_type', 'XOAUTH2')
+  -- credential taken from example here:
+  -- https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
+  msg:context_set(VCTX_MESS, 'outbound_smtp_auth_user', '[email protected]')
+  msg:context_set(VCTX_MESS, 'outbound_smtp_auth_pass', 'EwBAAl3BAAUFFpUAo7J3Ve0bjLBWZWCclRC3EoAA')
 end
 ```
 
 The following example shows how to use the new configuration variables to set distinct authorization parameters for two different domains.
 
-<a name="modules.outbound_smtp_auth.example.set_auth_parms"></a> 
+<a name="modules.outbound_smtp_auth.example.set_auth_parms"></a>
 
 
 ```
@@ -90,13 +100,17 @@ Domain "messagesystems.com" {
   Outbound_SMTP_AUTH_Type = "LOGIN"
   Outbound_SMTP_AUTH_User = "msys"
   Outbound_SMTP_AUTH_Pass = "msys"
-  Outbound_SMTP_AUTH_Key = "somestring"
 }
 
 Domain "sparkpost.com" {
   Outbound_SMTP_AUTH_Type = "PLAIN"
   Outbound_SMTP_AUTH_user = "sparkpost"
   Outbound_SMTP_AUTH_pass = "sparkpost"
-  Outbound_SMTP_AUTH_Key = "someotherstring"
 }
-```
+
+Domain "bird.com" {
+  Outbound_SMTP_AUTH_Type = "XOAUTH2"
+  Outbound_SMTP_AUTH_user = "[email protected]"
+  Outbound_SMTP_AUTH_pass = "EwBAAl3BAAUFFpUAo7J3Ve0bjLBWZWCclRC3EoAA"
+}
+```