|
2 | 2 |
|
3 | 3 | [](https://github.com/rdf-connect/template-processor-ts/actions/workflows/build-test.yml) |
4 | 4 |
|
5 | | -This repository contains a fully functioning dummy implementation of a logging processor written in TypeScript, meant to be used as a template in order to kickstart the development of your next processor. |
| 5 | +Template processor to kickstart the development of your next JavaScript/TypeScript RDF-Connect processor. |
6 | 6 |
|
7 | | -## Features |
| 7 | +## Usage |
8 | 8 |
|
9 | | -At the time of writing, this repositories sets up the following boilerplate and tools for you. |
| 9 | +To use the TypeScript TemplateProcessor in your RDF-Connect pipeline, you need to have a pipeline configuration that includes the [rdfc:NodeRunner](https://github.com/rdf-connect/js-runner) (check out their documentation to find out how to install and configure it). |
10 | 10 |
|
11 | | -- Placeholder implementation of a processor which accepts input from a stream, logs it to the console, and pipes it back into the outgoing stream. |
12 | | -- An initial `processor.ttl` file. |
13 | | -- Vitest configuration unit testing, including GitHub Actions configuration. |
14 | | -- Renovate notifications for dependency updates. |
15 | | -- Convenient publishing to GitHub Packages for every new release. |
16 | | -- ESLint/Prettier linting and styling, including a Husky git hook in combination with lint-staged. |
17 | | -- The MIT license. |
| 11 | +Next, you can add the JS/TS TemplateProcessor to your pipeline configuration as follows: |
18 | 12 |
|
19 | | -## Installation |
| 13 | +```turtle |
| 14 | +@prefix rdfc: <https://w3id.org/rdf-connect#>. |
| 15 | +@prefix owl: <http://www.w3.org/2002/07/owl#>. |
20 | 16 |
|
| 17 | +# Import the processor |
| 18 | +<> owl:imports <./node_modules/@rdfc/template-processor-ts/processor.ttl>. |
| 19 | +
|
| 20 | +### Define the channels your processor needs |
| 21 | +<channel1> a rdfc:Writer, rdfc:Reader. |
| 22 | +<channel2> a rdfc:Writer, rdfc:Reader. |
| 23 | +
|
| 24 | +# Attach the processor to the pipeline under the NodeRunner |
| 25 | +# Add the `rdfc:processor <template>` statement under the `rdfc:consistsOf` statement of the `rdfc:NodeRunner` |
| 26 | +
|
| 27 | +# Define and configure the processor |
| 28 | +<template> a rdfc:TemplateProcessorJs; |
| 29 | + rdfc:reader <channel1>; |
| 30 | + rdfc:writer <channel2>". |
21 | 31 | ``` |
| 32 | + |
| 33 | +## Development |
| 34 | + |
| 35 | +To start developing your own JavaScript/TypeScript processor based on this template, you can click the "Use this template" button on the top right of this page to create a new repository based on this template. |
| 36 | + |
| 37 | +You can then clone your newly created repository and start implementing your own processor logic in the `src/index.ts` file. |
| 38 | + |
| 39 | +First, install the dependencies using the following command: |
| 40 | + |
| 41 | +```bash |
22 | 42 | npm install |
23 | | -npm run build |
24 | 43 | ``` |
25 | 44 |
|
26 | | -## Example |
| 45 | +You can run the tests using: |
27 | 46 |
|
28 | | -An example configuration of the processor can be found in the `example` directory. |
| 47 | +```bash |
| 48 | +npm test |
| 49 | +``` |
29 | 50 |
|
30 | | -You can run this example by executing the following command: |
| 51 | +You can build the project using: |
31 | 52 |
|
32 | 53 | ```bash |
33 | | -npx @rdfc/js-runner example/pipeline.ttl |
| 54 | +npm run build |
34 | 55 | ``` |
35 | 56 |
|
36 | | -To enable all debug logs, add `DEBUG=*` before the command: |
| 57 | +### Logging |
37 | 58 |
|
38 | | -```bash |
39 | | -DEBUG=* npx @rdfc/js-runner example/pipeline.ttl |
| 59 | +The JavaScript runner and processors use the `winston` logging library for logging. |
| 60 | +The JavasScript runner initiates a logger that is passed to each processor, allowing them to log messages at various levels (info, warn, error, debug). |
| 61 | +You can access this logger in your processor class code on the `this.logger` property. |
| 62 | +Here's an example of how to use the logger in a processor: |
| 63 | + |
| 64 | +```typescript |
| 65 | +import { Processor } from '@rdfc/js-runner' |
| 66 | + |
| 67 | +class MyProcessor extends Processor<MyProcessorArgs> { |
| 68 | + async init(this: MyProcessorArgs & this): Promise<void> { |
| 69 | + this.logger.info('I am initializing my processor!') |
| 70 | + } |
| 71 | + // ... |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +This logger is configured to forward log messages to the RDF-Connect logging system. |
| 76 | +This means you can view and manage these logs in the RDF-Connect logging interface, allowing for consistent log management across different components of your RDF-Connect pipeline. |
| 77 | + |
| 78 | +If you want to create a child logger for a subclass or submethod, you can do so using the `extendLogger` method. |
| 79 | +Here's an example: |
| 80 | + |
| 81 | +```typescript |
| 82 | +import { Processor, extendLogger } from '@rdfc/js-runner' |
| 83 | + |
| 84 | +class MyProcessor extends Processor<MyProcessorArgs> { |
| 85 | + async init(this: MyProcessorArgs & this): Promise<void> { |
| 86 | + const childLogger = extendLogger(this.logger, 'init') |
| 87 | + childLogger.debug('This is a debug message from init.') |
| 88 | + } |
| 89 | + // ... |
| 90 | +} |
40 | 91 | ``` |
| 92 | + |
| 93 | + |
| 94 | +### Project Structure |
| 95 | + |
| 96 | +``` |
| 97 | +template-processor-ts/ # Root directory of the project |
| 98 | +├── .github/ # CI/CD configuration files for GitHub Actions and Renovate dependency updates |
| 99 | +├── src/ # Source code directory |
| 100 | +│ └── index.ts # Contains the main logic for the JS/TS processor |
| 101 | +├── tests/ # Directory for unit tests |
| 102 | +│ ├── index.test.ts # Functional tests for the processor logic |
| 103 | +│ └── processor.test.ts # Processor initialization tests |
| 104 | +├── package.json # Project metadata and dependencies |
| 105 | +├── tsconfig.json # TypeScript configuration file |
| 106 | +└── processor.ttl # RDF schema for the processor, used for metadata and configuration |
| 107 | +``` |
| 108 | + |
| 109 | + |
| 110 | +### Next Steps |
| 111 | + |
| 112 | +You can start developing your own TypeScript processor by modifying the `src/index.ts` file. |
| 113 | +This file contains the main logic for the processor and is where you can implement your custom processing logic. |
| 114 | + |
| 115 | +You probably want to modify the next things to make this template your own: |
| 116 | +- Change the repository package name and description in the `package.json` file. |
| 117 | +- Change the processor class name in the `src/index.ts` file. |
| 118 | +- Update the `processor.ttl` file to reflect the new processor name and any additional metadata or parameters. |
| 119 | +- Install any additional dependencies you might need for your processor using `npm install <package-name>`. |
| 120 | +- Implement your custom processing logic in the `init`, `transform`, and `produce` methods of the processor class. |
| 121 | +- Add unit tests for your processor in the `tests/` directory to ensure your processor works as expected. |
| 122 | +- Update this README file to reflect your processor's functionality and usage instructions. |
0 commit comments