EOP-281: OpenARC module and APIs

content/momentum/4/4-lua-summary-table.md

@@ -174,6 +174,8 @@ This section contains tables of Lua functions. Click the function name for detai
 | [msys.unlock](/momentum/4/lua/ref-msys-unlock) – Releases a lock obtained via msys.lock | mutexname | msys | 4.0 | any |
 | [msys.validate.dk.get_responsible_domain](/momentum/4/lua/ref-msys-validate-dk-get-responsible-domain) – This function requires module "dk_validate". "msg" is a mail message. "ctx" is the validation context. It returns the responsible domain for the current message | msg, ctx | msys.validate.dk | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [msys.validate.dk.sign](/momentum/4/lua/ref-msys-validate-dk-sign) – Sign a message using a Domain Key | msg, ctx, options | msys.validate.dk | 4.0 | core_data_validation |
+| [msys.validate.openarc.sign](/momentum/4/lua/ref-msys-validate-openarc-sign) – Sign a message using OpenARC | msg, options, [ar] | msys.validate.openarc | 5.0 | core_post_final_validation |
+| [msys.validate.openarc.verify](/momentum/4/lua/ref-msys-validate-openarc-verify) – Verify ARC sets | msg | msys.validate.openarc | 5.0 | data_spool, data_spool_each_rcpt |
 | [msys.validate.opendkim.get_num_sigs](/momentum/4/lua/ref-msys-validate-opendkim-get-num-sigs) – Return the number of DKIM signatures | dkim | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [msys.validate.opendkim.get_sig](/momentum/4/lua/ref-msys-validate-opendkim-get-sig) – Get a signature from a DKIM object | dkim, [num] | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [msys.validate.opendkim.get_sig_canons](/momentum/4/lua/ref-msys-validate-opendkim-get-sig-canons) – Fetch the canonicalizers used for a DKIM signature | dkim_sig | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
@@ -186,7 +188,7 @@ This section contains tables of Lua functions. Click the function name for detai
 | [msys.validate.opendkim.get_sig_selector](/momentum/4/lua/ref-msys-validate-opendkim-get-sig-selector) – Fetch the selector associated with a DKIM signature | dkim_sig | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [msys.validate.opendkim.get_sig_signalg](/momentum/4/lua/ref-msys-validate-opendkim-get-sig-signalg) – Return the signing algorithm as a string | dkim_sig | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [msys.validate.opendkim.sign](/momentum/4/lua/ref-msys-validate-opendkim-sign) – Sign a message using OpenDKIM | msg, vctx, [options] | msys.validate.opendkim | 4.0 | core_final_validation |
-| [msys.validate.opendkim.verify](/momentum/4/lua/ref-msys-validate-opendkim-verify) – Verify an DKIM signature | m | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
+| [msys.validate.opendkim.verify](/momentum/4/lua/ref-msys-validate-opendkim-verify) – Verify an DKIM signature | msg | msys.validate.opendkim | 4.0 | data, data_spool, data_spool_each_rcpt |
 | [sess:request_add_header](/momentum/4/lua/ref-sess-request-add-header) – Set the header of an HTTP session | header, value, replace | msys.httpclnt | 4.0 | http_request_eval |
 | [sess:request_delete_header](/momentum/4/lua/ref-sess-request-delete-header) – Delete a header from an HTTP session | header | msys.httpclnt | 4.0 | http_request_eval |
 | [sess:request_finalize](/momentum/4/lua/ref-sess-request-finalize) – Finalize changes to an HTTP request | update | msys.httpclnt | 4.0 | http_request_eval |

content/momentum/4/hooks/core-post-final-validation.md

@@ -0,0 +1,84 @@
+---
+lastUpdated: "11/26/2024"
+title: "post_final_validation"
+description: "hook invoked after the normal final validation"
+---
+
+<a name="hooks.core.post_final_validation"></a>
+## Name
+
+post_final_validation — This hook is invoked after the normal
+[final_validation](/momentum/3/3-api/hooks-core-final-validation) hook
+
+## Synopsis
+
+`#include "hooks/core/final_validation.h"`
+
+`int core_post_final_validation(void closure, ec_message *msg, accept_construct *ac, valiate_context *ctx)`
+
+
+## Description
+
+This hook is invoked right after the
+[final_validation](/momentum/3/3-api/hooks-core-final-validation) hook. Its return value
+does not have significance for now.
+This hook is added as the absolute last point before writing the message into spool.
+It guarantees that operations implemented in this hook will happen after the operations done in
+`final_validtion`.
+No any message modification expected after this stage.
+To avoid undefined race between multiple implementations of the same hook, you shall only have up
+to one implementation for this hook.
+It's the recommended hook point for ARC signing/sealing.
+
+
+**Parameters**
+
+The parameters from this hook are the same as the ones for `final_validation` hook.
+
+<dl class="variablelist">
+
+<dt>closure</dt>
+
+<dd>
+
+A pointer to the closure function.
+
+</dd>
+
+<dt>msg</dt>
+
+<dd>
+
+A pointer to an ec_message struct. For documentation of this data structure see [“ec_message”](/momentum/3/3-api/structs-ec-message)
+
+</dd>
+
+<dt>ac</dt>
+
+<dd>
+
+The `accept_construct` struct. For documentation of this data structure see [“accept_construct”](/momentum/3/3-api/structs-accept-construct)
+
+</dd>
+
+<dt>ctx</dt>
+
+<dd>
+
+The `validate_context` struct. For documentation of this data structure see [“validate_context”](/momentum/3/3-api/structs-validate-context)
+
+</dd>
+
+</dl>
+
+**Return Values**
+
+This hook returns `int`, but for now the return value has no significance, i.e. it is not checked in
+the caller.
+
+**Threading**
+
+This hook will be called in any thread.
+
+
+<a name="idp45866720"></a>

content/momentum/4/hooks/index.md

@@ -1,6 +1,6 @@
 ---
-lastUpdated: "10/05/2021"
-title: "Category File"
+lastUpdated: "11/26/2024"
+title: "Hook Points and C Functions Reference"
 type: "custom"
 name: "Hook Points and C Functions Reference"
 description: "This chapter includes hook point and C function reference material that is specific to Momentum 4 Hook points and C functions that are common to Momentum 4 and Momentum 3 are provided in the Momentum 3 x documentation For hook points see the C API For C functions see the..."
@@ -11,6 +11,7 @@ description: "This chapter includes hook point and C function reference material
 |---------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|
 | [msg_gen_data_spool](/momentum/4/hooks/msg-gen-data-spool)                            | This hook is invoked after a message has been generated by the msg_gen module |
 | [config_rsrc_setup](/momentum/4/hooks/config-rsrc-setup)                              | Register a resource                                                           |
+| [core_post_final_validation](/momentum/4/hooks/core-post-final-validation)            | Same usage as but invoked right after the `core_final_validation` hook        |
 | [ec_config_rsrc_get](/momentum/4/apis-ec-config-rsrc-get)                             | Return a resource list blobject from the configuration system                 |
 | [ec_httpsrv_register_auth](/momentum/4/apis-ec-httpsrv-register-auth)                 | Register an HTTP handler for authenticating a URI                             |
 | [ec_httpsrv_request_local_address](/momentum/4/apis-ec-httpsrv-request-local-address) | Returns the local IP address from the current session                         |

content/momentum/4/lua/index.md

@@ -191,6 +191,8 @@ description: "This section details all Lua functions Functions are ordered alpha
 | [msys.tls_params.set](/momentum/4/lua/ref-msys-tls-params-set) | Set a tls parameter string on a per connection basis |
 | [msys.validate.dk.get_responsible_domain](/momentum/4/lua/ref-msys-validate-dk-get-responsible-domain) | Return the domain responsible for the current message |
 | [msys.validate.dk.sign](/momentum/4/lua/ref-msys-validate-dk-sign) | Sign a message using a Domain Key |
+| [msys.validate.opendarc.sign](/momentum/4/lua/ref-msys-validate-opendarc-sign) | Sign a message using OpenARC |
+| [msys.validate.opendarc.verify](/momentum/4/lua/ref-msys-validate-opendarc-verify) | Verify ARC sets |
 | [msys.validate.opendkim.get_num_sigs](/momentum/4/lua/ref-msys-validate-opendkim-get-num-sigs) | Return the number of DKIM signatures |
 | [msys.validate.opendkim.get_sig](/momentum/4/lua/ref-msys-validate-opendkim-get-sig) | Get a signature from a DKIM object |
 | [msys.validate.opendkim.get_sig_canons](/momentum/4/lua/ref-msys-validate-opendkim-get-sig-canons) | Fetch the canonicalizers used for a signature |

content/momentum/4/lua/ref-msys-validate-openarc-sign.md

@@ -0,0 +1,125 @@
+---
+lastUpdated: "11/26/2024"
+title: "msys.validate.openarc.sign"
+description: "msys validate openarc sign seal add ARC set headers"
+---
+
+<a name="lua.ref.msys.validate.openarc.sign"></a>
+## Name
+
+msys.validate.openarc.sign — builds and adds the ARC set headers into an email.
+
+msys.validate.openarc.seal - synonym of `msys.validation.openarc.sign`.
+
+## Synopsis
+
+`msys.validate.openarc.verify(msg, options, ar)`
+
+```
+msg: userdata, ec_message type
+options: table
+ar: string, optional. It's the message's authentication assessment to be enshrined into the AAR header.
+
+```
+
+## Description
+
+This function does ARC validation first, then combine the validation result with authentication
+assessments from other methods (e.g. SPF, DKIM, etc) defined by the `ar` and put it into the AAR
+(ARC-Authentication-Results) header;
+ then sign and seal the message by adding the AMS (ARC-Message-Signature) and AS
+(ARC-Seal) headers, using the signing mechanism defined in the `options` table.
+
+This function requires the [`openarc`](/momentum/4/modules/openarc) module.
+
+Enable this function with the statement `require('msys.validate.openarc')`.
+
+This function takes the following parameters:
+
+*   `msg` - mail message to sign
+
+*   `options`   - table defines the options for signature generation/signing:
+
+    *   `signing_domain` – signing domain
+
+    *   `selector` – signing selector
+
+    *   `authservid` – authentication service identifier, as
+        [authserv-id](https://datatracker.ietf.org/doc/html/rfc8601#section-2.5) defined in RFC.
+
+        If not set, will be default to the hostname.
+
+    *   `header_canon` – header canonicalization setting.
+
+        Supported values are `relaxed`, `simple`. Default to `relaxed`.
+
+    *   `body_canon` – body canonicalization setting
+
+        Supported values are `relaxed`, `simple`. Default to `relaxed`.
+
+    *   `digest` – signing algorithm digest setting.
+
+        Supported values are `rsa-sha1` and `rsa-sha256`. Default to `rsa-sha256`.
+
+    *   `keyfile` – signing key file
+
+    *   `keybuf` – signing key
+
+        Must contain the PEM encoded private key to use for signing the
+        message. This must be a contiguous string, with no line breaks and no white spaces, without the
+        `BEGIN` and `END` tags that are found in the key file itself. The format is similar to the
+        format used for OpenDKIM signing.
+
+        If not defined, will be built from the `keyfile`.
+
+    *   `headerlist` – "`;`" separated list of headers to sign
+
+    *   `oversign_headerlist` – "`;`" seperated list of headers for over signing
+
+    *   `skip_ar_header_update` – if set, no update to the AR (Authentication-Results) header.
+
+        If not set, Momentum will append the ARC verification result (e.g. `arc=pass`) to
+        the existing AR header or create one if it does not exist.
+
+*   `ar` - authentication assessment to be enshrined into the AAR (ARC-Authentication-Results) header.
+
+    If not provided, Momentum will take the value from the existing `Authentication-Results` header.
+    Momentum appends this value with the ARC verification result (e.g. `arc=pass`) and use it to
+    build the AAR header.
+
+
+### Note
+
+Since ARC sealing should happen after all potential modification of a message is done, this function
+shall be invoked in the `post_final_validation` stage after all the other validation phases.
+
+If for any reason the ARC signing/sealing failed, the context variable `arc_cv` of the `ec_message`
+will not be set, and the error reason is logged into paniclog.
+
+
+<a name="lua.ref.msys.validate.opendarc.sign.example"></a>
+### Example
+
+
+```
+require("msys.core");
+require("msys.validate.openarc");
+local mod = {};
+
+function mod:core_post_final_validation(msg, accept, vctx)
+  local sealer = {}
+  sealer.signing_domain = "sparkpost.com"
+  sealer.selector = "dkim-s1024"
+  sealer.keyfile = "path-to-keyfile"
+  sealer.headerlist = "From:Subject:Date:To:MIME-Version:Content-Type"
+  sealer.oversign_headerlist = "From:To:Subject"
+
+  msys.validate.openarc.sign(msg, sealer)
+end
+
+msys.registerModule("openarc_sign", mod);
+```
+
+## See Also
+
+[msys.validate.opendarc.verify](/momentum/4/lua/ref-msys-validate-openarc-verify)

content/momentum/4/lua/ref-msys-validate-openarc-verify.md

@@ -0,0 +1,62 @@
+---
+lastUpdated: "11/26/2024"
+title: "msys.validate.openarc.verify"
+description: "msys validate openarc verify Verify ARC sets headers"
+---
+
+<a name="lua.ref.msys.validate.openarc.verify"></a>
+## Name
+
+msys.validate.openarc.verify — Verifies ARC set headers in an email, stores the verification results
+(`none/pass/fail`) into the email's context variable.
+
+## Synopsis
+
+`msys.validate.openarc.verify(msg)`
+
+`msg: userdata, ec_message type`<a name="idp19138336"></a>
+## Description
+
+This function validates the ARC set headers contained in the input message. The validation result
+will be stored as string value (`none` or `pass` or `fail`) in the `ec_message`'s context variable
+of `arc_cv`. A caller can take actions (e.g. disposition of the message) based on the validation
+result.
+
+This function requires the [`openarc`](/momentum/4/modules/openarc) module.
+
+Enable this function with the statement `require('msys.validate.openarc')`.
+
+### Note
+
+If the `ec_message` context variable `arc_cv` is not set after this function call, errors happened
+and logged into paniclog.
+
+This function invokes dns lookup for signature validation. It's recommended to invoke it from a hook
+which would not block Momentum's main tasks, e.g. from the `validate_data_spool` hook.
+
+<a name="lua.ref.msys.validate.opendarc.verify.example"></a>
+### Example
+
+
+```
+require("msys.core");
+require("msys.extended.message");
+require("msys.validate.openarc");
+local mod = {};
+
+function mod:validate_data_spool(msg, ac, vctx)
+  msys.validate.openarc.verify(msg)
+  local cv = msg:context_get(msys.core.ECMESS_CTX_MESS, "arc_cv")
+  if cv then
+    print("ARC validation result: ", cv)
+  else
+	print("Failed to do ARC validation. Check paniclog for reasons.")
+  end
+end
+
+msys.registerModule("openarc_verify", mod);
+```
+
+## See Also
+
+[msys.validate.opendarc.sign](/momentum/4/lua/ref-msys-validate-openarc-sign)

content/momentum/4/modules/index.md

@@ -60,6 +60,7 @@ description: "Table of Contents 71 1 Introduction 71 2 ac auth Authentication Ha
 | [msgc](/momentum/4/modules/msgc) | Message Systems Group Communication |
 | [msg_gen](/momentum/4/modules/msg-gen) | Message Generation |
 | [mxip](/momentum/4/modules/mxip) | IP Addresses In MX Records |
+| [openarc](/momentum/4/modules/openarc) | Open Source ARC |
 | [opendkim](/momentum/4/modules/opendkim) | Open Source DKIM |
 | [outbound_audit](/momentum/4/modules/outbound-audit) | Outbound traffic analytics |
 | [outbound_smtp_auth(modules.outbound_smtp_auth.php) |

content/momentum/4/modules/openarc.md

@@ -0,0 +1,41 @@
+---
+lastUpdated: "11/26/2024"
+title: "openarc – Open Source ARC"
+description: "The openarc module wraps the open source libopenarc API into Momentum to provide Momentum APIs to do ARC validation verification signature signing and sealing..."
+---
+
+ARC is an acronym for Authenticated Received Chain. It’s a technology protocol defined in
+[RFC8617](https://datatracker.ietf.org/doc/html/rfc8617). It provides an authenticated "chain of
+custody" for a message, allowing each entity that handles the message to see what entities handled
+it before and what the message's authentication assessment was at each step in the handling flow.
+
+The openarc module add ARC capability to Momentum. It provides Lua APIs for
+ARC validation on a received email, and ARC siging and sealing on an outgoing email.
+When the module is enabled, ARC validation and signing/sealing can be achieved through calling these APIs from hook policies.
+
+
+### <a name="modules.openarc.configuration"></a> Configuration
+
+You need to enable the openarc module in the ecelerity configuration file to use the feature:
+
+```
+openarc {}
+```
+
+### Lua APIs and examples
+
+[msys.validate.opendarc.verify](/momentum/4/lua/ref-msys-validate-openarc-verify)
+
+[msys.validate.opendarc.sign](/momentum/4/lua/ref-msys-validate-openarc-sign)
+
+
+### Hook points to invoke the APIs
+
+The `msys.validate.opendarc.sign` does verification first. You should only invoke one of the APIs,
+    either `verify` or `sign` but not both.
+It's recommended to invoke `msys.validate.opendarc.verify` in `validate_data_spool` or
+[`validate_data_spool_each_rcpt`](/momentum/3/3-api/hooks-core-validate-data-spool-each-rcpt) hook.
+`msys.validate.opendarc.sign` shall be invoked in the last validation phase, in
+[`post_final_validation`](/momentum/4/hooks/core-post-final-validation) hook.
+
+See API examples for hook usages.

content/momentum/4/modules/summary-all-modules.md

@@ -70,6 +70,7 @@ All modules are listed alphabetically with a brief description. Singleton module
 | [msgc_client](/momentum/4/modules/msgc) (*singleton*) | 4.0 | The client component of MSGC |   |  ✓ |   |   |
 | [msgc_server](/momentum/4/modules/msgc) (*singleton*) | 4.0 | The server component of MSGC |   |  ✓ |   |   |
 | [“mxip - IP Addresses In MX Records”](/momentum/4/modules/mxip) | 4.2 | Enable Momentum to deliver to domains with a textual IP address |   |   |   |   |
+| [“openarc – Open Source ARC”](/momentum/4/modules/openarc) | 5.0 | Validate/sign mail using ARC |   |   |  ✓ |   |
 | [“opendkim – Open Source DKIM”](/momentum/4/modules/opendkim) | 4.0 | Validate/sign mail using DKIM signatures |   |   |  ✓ |   |
 | [“outbound_audit – Outbound traffic analytics”](/momentum/4/modules/outbound-audit) | 4.0 | Provides time-series analytics on the behavior of receiving domains |   |  ✓ |   |   |
 | [“outbound_smtp_auth”](/momentum/4/modules/outbound-smtp-auth) | 4.2 | Enables users to specify authentication parameters for a given set of messages |   |   |   |   |