sparql-ingest-processor-ts

TypeScript RDF-Connect processor for ingesting SDS records into a SPARQL endpoint.

This processor takes a stream of RDF records, transforms them into SPARQL Update queries, and executes them against a SPARQL Graph Store via the SPARQL Protocol.
It supports INSERT DATA, DELETE INSERT WHERE, and DELETE WHERE queries, configurable through change semantics or SDS record content.

---

Usage

Installation

npm install
npm run build

Or install from NPM:

npm install @rdfc/sparql-ingest-processor-ts

---

Pipeline Configuration Example

@prefix rdfc: <https://w3id.org/rdf-connect#>.
@prefix owl: <http://www.w3.org/2002/07/owl#>.

### Import the processor definitions
<> owl:imports <./node_modules/@rdfc/sparql-ingest-processor-ts/processors.ttl>.

### Define the channels your processor needs
<in> a rdfc:Reader.
<out> a rdfc:Writer.

### Attach the processor to the pipeline under the NodeRunner
# Add the `rdfc:processor <ingester>` statement under the `rdfc:consistsOf` statement of the `rdfc:NodeRunner`

### Define and configure the processor
<ingester> a rdfc:SPARQLIngest;
    rdfc:memberStream <in>;
    rdfc:ingestConfig [
        rdfc:memberIsGraph false;
        rdfc:memberShape "http://ex.org/Shape1", "http://ex.org/Shape2";
        rdfc:changeSemantics [
            rdfc:changeTypePath "http://ex.org/changeType";
            rdfc:createValue "http://ex.org/Create";
            rdfc:updateValue "http://ex.org/Update";
            rdfc:deleteValue "http://ex.org/Delete"
        ];
        rdfc:targetNamedGraph "http://ex.org/myGraph";
        rdfc:transactionConfig [
            rdfc:transactionIdPath "http://ex.org/transactionId";
            rdfc:transactionEndPath "http://ex.org/transactionEnd"
        ];
        rdfc:graphStoreUrl "http://example.org/sparql";
        rdfc:forVirtuoso false
    ];
    rdfc:sparqlWriter <out>.

---

Configuration

Parameters of rdfc:SPARQLIngest:

• rdfc:memberStream (rdfc:Reader, required): Input SDS record stream.
• rdfc:ingestConfig (rdfc:IngestConfig, required): Configuration for ingest behavior.
• rdfc:sparqlWriter (rdfc:Writer, optional): Output stream of generated SPARQL queries.

---

Parameters of rdfc:IngestConfig:

• rdfc:memberIsGraph (boolean, required): Whether each SDS record represents a named graph.
• rdfc:memberShape (string, optional, repeatable): SHACL shape identifiers used to guide query construction when payloads are incomplete.
• rdfc:changeSemantics (rdfc:ChangeSemantics, optional): Configures mapping between change types (create/update/delete) and SPARQL operations.
• rdfc:targetNamedGraph (string, optional): Force all operations into a specific named graph (ignored if memberIsGraph = true).
• rdfc:transactionConfig (rdfc:TransactionConfig, optional): Groups records by transaction ID for atomic updates.
• rdfc:graphStoreUrl (string, optional): SPARQL Graph Store endpoint URL.
• rdfc:forVirtuoso (boolean, optional): Enables Virtuoso-specific handling.
• rdfc:accessToken (string, optional): Access token for authenticated graph stores.
• rdfc:measurePerformance (rdfc:PerformanceConfig, optional): Enables performance measurement of SPARQL queries.

---

Parameters of rdfc:ChangeSemantics:

• rdfc:changeTypePath (string, required): Predicate identifying the type of change in SDS records.
• rdfc:createValue (string, required): Value representing a create operation.
• rdfc:updateValue (string, required): Value representing an update operation.
• rdfc:deleteValue (string, required): Value representing a delete operation.

---

Parameters of rdfc:TransactionConfig:

• rdfc:transactionIdPath (string, required): Predicate identifying the transaction ID.
• rdfc:transactionEndPath (string, required): Predicate marking the last record in a transaction.

---

Parameters of rdfc:PerformanceConfig:

• rdfc:name (string, required): Name of the performance measurement run.
• rdfc:outputPath (string, required): File path where performance logs will be written.
• rdfc:failureIsFatal (boolean, optional): If true, aborts on performance measurement failure.
• rdfc:queryTimeout (integer, optional): Maximum query execution time in milliseconds.

---

Example

<ingester> a rdfc:SPARQLIngest;
    rdfc:memberStream <in>;
    rdfc:ingestConfig [
        rdfc:memberIsGraph true;
        rdfc:targetNamedGraph "http://example.org/targetGraph";
        rdfc:graphStoreUrl "http://example.org/sparql"
    ];
    rdfc:sparqlWriter <out>.

---

Notes

• Delete operations can be handled differently depending on how complete the SDS record payload is.
• When memberIsGraph = true, queries are wrapped with GRAPH and WITH clauses.
• Transactions can buffer multiple SDS records and commit them together using rdfc:transactionConfig.
• SHACL shapes (rdfc:memberShape) can be provided to help identify deletion targets when payloads are incomplete.