`use_shapes` error with properties

[multimeric]

There's currently a bit of an issue with the use_shapes argument to validate(). It seems like this causes shapes with IDs to be removed from the graph unless they're included in the list of shapes. This logic makes sense except that it doesn't remove the triples that reference the removed shapes.

For example, if we have a node shape with property shapes A and B and we ask for pySHACL to use only the node shape and property A, it still retains the node shape's link to B which is gone. This results in a SHACL PropertyShape not found error. Example below.

I think this is outside of the SHACL spec (?) so we can do anything sensible here. I would suggest removing all triples that have the removed shape as an object as well as subject in this scenario.

---

from rdflib.plugins.parsers.jsonld import to_rdf
from rdflib import Graph
import pyshacl

shapes_graph = to_rdf({
    "@context": {
        "ex": "http://example.org/",
        "sh": "http://www.w3.org/ns/shacl#"
    },
    "@graph": [
        {
            "@id": "ex:PersonShape",
            "@type": "sh:NodeShape",
            "sh:targetClass": {"@id": "ex:Person"},
            "sh:property": [
                {
                    "@id": "ex:NameProperty",
                    "sh:path": {"@id": "ex:name"},
                    "sh:minCount": 1
                },
                {
                    "@id": "ex:AgeProperty",
                    "sh:path": {"@id": "ex:age"},
                    "sh:minInclusive": 18
                }
            ]
        }
    ]
}, Graph())

data_graph = to_rdf({
    "@context": {
        "ex": "http://example.org/",
    },
    "@id": "ex:person1",
    "@type": "ex:Person",
    "ex:name": "John Doe",
    "ex:age": 25
}, Graph())

result = pyshacl.validate(data_graph, shacl_graph=shapes_graph, use_shapes=[
    "http://example.org/PersonShape",
    "http://example.org/NameProperty"
])

Traceback (most recent call last):
  File "/Users/milton.m/Programming/MerscopeMetadata/use_shapes.py", line 41, in <module>
    result = pyshacl.validate(data_graph, shacl_graph=shapes_graph, use_shapes=[
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/MerscopeMetadata/.venv/lib/python3.12/site-packages/pyshacl/entrypoints.py", line 178, in validate
    conforms, report_graph, report_text = validator.run()
                                          ^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/MerscopeMetadata/.venv/lib/python3.12/site-packages/pyshacl/validator.py", line 315, in run
    _is_conform, _reports = s.validate(executor, g, focus=on_focus_nodes)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/MerscopeMetadata/.venv/lib/python3.12/site-packages/pyshacl/shape.py", line 772, in validate
    _is_conform, _reports = c.evaluate(executor, target_graph, focus_value_nodes, _e_p_copy)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/MerscopeMetadata/.venv/lib/python3.12/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 99, in evaluate
    _nc, _r = self._evaluate_property_shape(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/MerscopeMetadata/.venv/lib/python3.12/site-packages/pyshacl/constraints/core/shape_based_constraints.py", line 116, in _evaluate_property_shape
    raise ReportableRuntimeError(
pyshacl.errors.ReportableRuntimeError: SHACL PropertyShape not found: The shape referenced by sh:property does not exist. Please check if the shape 'None' is defined.

pyshacl==0.30.1
rdflib==7.1.4

[ashleysommer]

Hi @multimeric
Thanks for reporting this.

In my tests, the property shapes attached to a nodeshape are usually BNodes. In that case, PySHACL detects they are dependents of the NodeShape, and includes them in the list of included shapes in memory.

However when the dependent property shapes are named nodes, it is up to the user to include them in their list of "use_shapes" (like you have with http://example.org/NameProperty).
I didn't consider what would happen in the case that a dependent shape is deliberately omitted. This is an interesting case.

Normally what I would do in this situation is have a different shape just for this purpose:

shapes_graph = to_rdf({
    "@context": {
        "ex": "http://example.org/",
        "sh": "http://www.w3.org/ns/shacl#"
    },
    "@graph": [
        {
            "@id": "ex:PersonShapeNameOnly",
            "@type": "sh:NodeShape",
            "sh:targetClass": {"@id": "ex:Person"},
            "sh:property": {
                  "@id": "ex:NameProperty",
                  "sh:path": {"@id": "ex:name"},
                  "sh:minCount": 1
              }
        }
    ]
}, Graph())

result = pyshacl.validate(data_graph, shacl_graph=shapes_graph, use_shapes=[
    "http://example.org/PersonShapeNameOnly",
    "http://example.org/NameProperty"
])

Your usage does expose this bug though

pyshacl.errors.ReportableRuntimeError: SHACL PropertyShape not found: The shape referenced by sh:property does not exist. Please check if the shape 'None' is defined.

I believe the list of property shapes on a NodeShape should never contain a None entry, even when named property shapes are omitted from the use_shapes list. So there is a bug to fix there.

I suppose one easy way to accommodate this use case, and work around that bug, is to simply ignore/skip any shapes in the list that are None. That might however be unexpected behaviour for most users, eg if they forget to add a PropertyShape into the use_shapes list, there will be no indication that shape is silently skipped by the enclosing NodeShape.

[multimeric]

Ideally I would rather not have to make a new shape here, because then I have to duplicate the NodeShape logic.

I suppose one easy way to accommodate this use case, and work around that bug, is to simply ignore/skip any shapes in the list that are None

Can't we instead tweak the underlying logic that is creating the None? What do you think about my proposed logic which is to remove all triples that have the untargeted nodes as subjects or objects?

[ashleysommer]

Can't we instead tweak the underlying logic that is creating the None?

Yes, that is a better solution, and is the correct fix, it is what I will do. However I mentioned the simper fix as a fallback, to prevent this same issue from causing problems again in the future.