Allow passing around user data inside a Policy object.

[deivse]

As discussed here #12360 (comment)
This PR is mostly changes to the cryptography-x509-verification module. I will do the PyPolicy implementation and attribute moving next.

[deivse]

PS: waiting for coverage to fail so I can fix it.

[deivse]

Nevermind, coverage is still 100%. There's some issue with CI on alpine though.

[alex]

Are teh separate PolicyDefinition and Policy types required, or could we possibly just store the PolicyExtra directly on the Policy?

[deivse]

Are teh separate PolicyDefinition and Policy types required, or could we possibly just store the PolicyExtra directly on the Policy?

I think it is required but correct me if I'm wrong. The reasoning is that in our current usecase, PolicyExtra is the object that actually owns Policy's memory. So if PolicyExtra was to be a field on Policy, we would need to somehow create a struct whose lifetime is defined by the lifetime of it's member. Which I don't think is possible directly. There might be some solution using unsafe but I don't think it's warranted. While the current solution might raise some eyebrows, I think it's the lesser evil here.

Edit: I did spent quite some time trying to get something like this working yesterday, but decided that this is the better approach (and the one I could actually get compiling 😄)

[alex]

Here's a basic sketch of what I was thinking. I don't think lifetimes should be a problem, but does it look like it does what's required?

diff --git a/src/rust/cryptography-x509-verification/src/certificate.rs b/src/rust/cryptography-x509-verification/src/certificate.rs
index ec1dd33a8..4c9fc256b 100644
--- a/src/rust/cryptography-x509-verification/src/certificate.rs
+++ b/src/rust/cryptography-x509-verification/src/certificate.rs
@@ -55,6 +55,7 @@ Xw4nMqk=
         type Key = ();
         type Err = ();
         type CertificateExtra = ();
+        type PolicyExtra = ();
 
         fn public_key(&self, _cert: &Certificate<'_>) -> Result<Self::Key, Self::Err> {
             // Simulate failing to retrieve a public key.
diff --git a/src/rust/cryptography-x509-verification/src/ops.rs b/src/rust/cryptography-x509-verification/src/ops.rs
index 47142dc85..c8e8a5714 100644
--- a/src/rust/cryptography-x509-verification/src/ops.rs
+++ b/src/rust/cryptography-x509-verification/src/ops.rs
@@ -72,6 +72,9 @@ pub trait CryptoOps {
     /// Extra data that's passed around with the certificate.
     type CertificateExtra;
 
+    /// Extra data that's passed around with the policy.
+    type PolicyExtra;
+
     /// Extracts the public key from the given `Certificate` in
     /// a `Key` format known by the cryptographic backend, or `None`
     /// if the key is malformed.
diff --git a/src/rust/cryptography-x509-verification/src/policy/extension.rs b/src/rust/cryptography-x509-verification/src/policy/extension.rs
index f950d5aba..0983fb4ba 100644
--- a/src/rust/cryptography-x509-verification/src/policy/extension.rs
+++ b/src/rust/cryptography-x509-verification/src/policy/extension.rs
@@ -604,6 +604,7 @@ mod tests {
         let ops = PublicKeyErrorOps {};
         let policy = Policy::server(
             ops,
+            (),
             Subject::DNS(DNSName::new("example.com").unwrap()),
             epoch(),
             None,
@@ -644,6 +645,7 @@ mod tests {
         let ops = PublicKeyErrorOps {};
         let policy = Policy::server(
             ops,
+            (),
             Subject::DNS(DNSName::new("example.com").unwrap()),
             epoch(),
             None,
@@ -678,6 +680,7 @@ mod tests {
         let ops = PublicKeyErrorOps {};
         let policy = Policy::server(
             ops,
+            (),
             Subject::DNS(DNSName::new("example.com").unwrap()),
             epoch(),
             None,
@@ -709,6 +712,7 @@ mod tests {
         let ops = PublicKeyErrorOps {};
         let policy = Policy::server(
             ops,
+            (),
             Subject::DNS(DNSName::new("example.com").unwrap()),
             epoch(),
             None,
@@ -738,6 +742,7 @@ mod tests {
         let ops = PublicKeyErrorOps {};
         let policy = Policy::server(
             ops,
+            (),
             Subject::DNS(DNSName::new("example.com").unwrap()),
             epoch(),
             None,
diff --git a/src/rust/cryptography-x509-verification/src/policy/mod.rs b/src/rust/cryptography-x509-verification/src/policy/mod.rs
index 1866fa0a8..5b9d471e5 100644
--- a/src/rust/cryptography-x509-verification/src/policy/mod.rs
+++ b/src/rust/cryptography-x509-verification/src/policy/mod.rs
@@ -200,6 +200,8 @@ impl Subject<'_> {
 pub struct Policy<'a, B: CryptoOps> {
     pub ops: B,
 
+    pub extra: B::PolicyExtra,
+
     /// A top-level constraint on the length of intermediate CA paths
     /// constructed under this policy.
     ///
@@ -237,6 +239,7 @@ pub struct Policy<'a, B: CryptoOps> {
 impl<'a, B: CryptoOps> Policy<'a, B> {
     fn new(
         ops: B,
+        extra: B::PolicyExtra,
         subject: Option<Subject<'a>>,
         time: asn1::DateTime,
         max_chain_depth: Option<u8>,
@@ -244,6 +247,7 @@ impl<'a, B: CryptoOps> Policy<'a, B> {
     ) -> Self {
         Self {
             ops,
+            extra,
             max_chain_depth: max_chain_depth.unwrap_or(DEFAULT_MAX_CHAIN_DEPTH),
             subject,
             validation_time: time,
@@ -346,9 +350,15 @@ impl<'a, B: CryptoOps> Policy<'a, B> {
     /// **IMPORTANT**: This is **not** the appropriate API for verifying
     /// website (i.e. server) certificates. For that, you **must** use
     /// [`Policy::server`].
-    pub fn client(ops: B, time: asn1::DateTime, max_chain_depth: Option<u8>) -> Self {
+    pub fn client(
+        ops: B,
+        extra: B::PolicyExtra,
+        time: asn1::DateTime,
+        max_chain_depth: Option<u8>,
+    ) -> Self {
         Self::new(
             ops,
+            extra,
             None,
             time,
             max_chain_depth,
@@ -360,12 +370,14 @@ impl<'a, B: CryptoOps> Policy<'a, B> {
     /// defined in the CA/B Forum's Basic Requirements.
     pub fn server(
         ops: B,
+        extra: B::PolicyExtra,
         subject: Subject<'a>,
         time: asn1::DateTime,
         max_chain_depth: Option<u8>,
     ) -> Self {
         Self::new(
             ops,
+            extra,
             Some(subject),
             time,
             max_chain_depth,
diff --git a/src/rust/src/x509/verify.rs b/src/rust/src/x509/verify.rs
index a1be8484f..cfd2efa94 100644
--- a/src/rust/src/x509/verify.rs
+++ b/src/rust/src/x509/verify.rs
@@ -25,6 +25,8 @@ impl CryptoOps for PyCryptoOps {
     type Key = pyo3::Py<pyo3::PyAny>;
     type Err = CryptographyError;
     type CertificateExtra = pyo3::Py<PyCertificate>;
+    // TODO: This will be `Py<PyPolicy>` once there is a Python policy type.
+    type PolicyExtra = pyo3::Py<pyo3::PyAny>;
 
     fn public_key(&self, cert: &Certificate<'_>) -> Result<Self::Key, Self::Err> {
         pyo3::Python::with_gil(|py| -> Result<Self::Key, Self::Err> {
@@ -158,7 +160,7 @@ impl PolicyBuilder {
         };
 
         // TODO: Pass extension policies here once implemented in cryptography-x509-verification.
-        let policy = Policy::client(PyCryptoOps {}, time, self.max_chain_depth);
+        let policy = Policy::client(PyCryptoOps {}, py.None(), time, self.max_chain_depth);
 
         Ok(PyClientVerifier { policy, store })
     }
@@ -191,6 +193,7 @@ impl PolicyBuilder {
             // TODO: Pass extension policies here once implemented in cryptography-x509-verification.
             Ok::<PyCryptoPolicy<'_>, pyo3::PyErr>(Policy::server(
                 PyCryptoOps {},
+                py.None(),
                 subject,
                 time,
                 self.max_chain_depth,

[deivse]

Well the problems I was having are related to this part:

let policy = OwnedPolicy::try_new(subject_owner, |subject_owner| {
            let subject = build_subject(py, subject_owner)?;

            // TODO: Pass extension policies here once implemented in cryptography-x509-verification.
            Ok::<PyCryptoPolicy<'_>, pyo3::PyErr>(Policy::server(
                PyCryptoOps {},
                py.None(),
                subject,
                time,
                self.max_chain_depth,
            ))
        })?;

Here we construct a policy and pass py.None in there which is fine. But in the actual use case we want something like this:

        let policy = OwnedPolicy::try_new(subject_owner, |subject_owner| {
            let subject = build_subject(py, subject_owner)?;

            // TODO: Pass extension policies here once implemented in cryptography-x509-verification.
            Ok::<PyCryptoPolicy<'_>, pyo3::PyErr>(Policy::server(
                PyCryptoOps {},
                py.None(),
                subject,
                time,
                self.max_chain_depth,
            ))
        })?;
        policy.with_dependent_mut(|owner, dep| dep.extra = PyPolicy(policy));

This is of course invalid. I've tried a couple variations of this and haven't gotten a working solution (although maybe someone smarter/more experienced with rust could.) The only solution I have right now (except the separate struct for PolicyDefinition) is to create two identical OwnedPolicies and make a PyPolicy from the second one, which is then assigned to the first one's extra field:

       let mut policy = OwnedPolicy::try_new(subject_owner, |subject_owner| {
   		// create policy...
       })?;
       let policy2 = OwnedPolicy::try_new(subject_owner, |subject_owner| {
   		// create identical policy...
       })?;
       policy.with_dependent_mut(|owner, dep| dep.extra = PyPolicy(policy2));

Might be a better way to do this, but I couldn't really figure it out.

[deivse]

When it's a separate struct, this problem doesn't exist because the PyClient/ServerVerifier just holds a PyPolicyDefinition which is just

#[pyo3::pyclass(module = "cryptography.x509.verification", name = "Policy", frozen)]
pub(crate) struct PyPolicyDefinition(pub(super) OwnedPolicyDefinition);

And the actual Policy is constructed only in the verification function with the reference the lifetime of which is dependant on the lifetime of the Verifier itself (which is fine since this Policy only needs to live inside the function's scope):

let policy = Policy::new(
            self.py_policy.get().0.borrow_dependent(),
            self.py_policy.clone_ref(py),
);

[deivse]

Not saying it's necessarily impossible, just saying what I struggled with. I'd be happy to learn if you can come up with a better solution.

[alex]

Got it, thanks for explaining.