Build

Introduction:

Source code...for those of you who want more depth...to ply the inner workings of OpenRefine. You still need to install it to test and debug any modifications, so here are those complete instructions.

Also, see the Java version requirements on the Install page.

NOTE: If you have previously installed the extension, you will need to replace it in the extensions directory with the newly built version, e.g., delete rdf-transform directory in the extensions directory and unzip the new file there. See the IDE Support section below for automating this task.

Building From Source:

TL;DR:

In general, the Long version will be needed for new installs.

In all cases, a JDK version must be installed and used as the default Java development version. Examples:

sudo apt install openjdk-17-jdk openjdk-17-jdk-headless openjdk-17-dbg openjdk-17-jre openjdk-17-jre-headless openjdk-17-doc
sudo apt install openjdk-21-jdk openjdk-21-jdk-headless openjdk-21-dbg openjdk-21-jre openjdk-21-jre-headless openjdk-21-doc

update-java-alternatives --list
sudo update-java-alternatives --set java-1.17.0-openjdk-amd64

Short:

git clone https://github.com/AtesComp/rdf-transform
cd rdf-transform
mvn clean
mvn dependency:resolve -U
mvn compile
mvn assembly:single
rm -rf ~/.local/share/openrefine/extensions/rdf-transform*
unzip target/rdf-transform-$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout).zip -d ~/.local/share/openrefine/extensions
~/path/to/openrefine/refine

Long:

mkdir OpenRefineExtensions
cd OpenRefineExtensions
git clone https://github.com/AtesComp/rdf-transform
cd ..
git clone https://github.com/OpenRefine/OpenRefine
cd OpenRefine
./refine clean
./refine build
mvn package -DskipTests=true

cd ../OpenRefineExtensions/rdf-transform
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/main/webapp/WEB-INF/lib/openrefine-main.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/modules/core/target/openrefine-core.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/modules/grel/target/openrefine-grel.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
mvn clean
mvn dependency:resolve -U
mvn compile
mvn assembly:single
rm -rf ~/.local/share/openrefine/extensions/rdf-transform*
unzip target/rdf-transform-$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout).zip -d ~/.local/share/openrefine/extensions
cd ../../OpenRefine
./refine

Short Steps

A local project repository (see the "project-repository" directory) contains an OpenRefine jar file ready for use by the maven compile process. If you want or need to compile OpenRefine, see the Long Steps below to create the OpenRefine jar file.

1. From some top level development directory, create a local repository for this RDF Transform extension:
   • Clone the extension at the top level development directory where you want the /rdf-transform sub-directory:
     • git clone https://github.com/AtesComp/rdf-transform
2. Compile the RDF Transform extension:
   • Change directories to the RDF Transform extension:
     • cd rdf-transform
   • Clean the extension:
     • mvn clean
   • Update extension's dependencies:
     • mvn dependency:resolve -U
   • Compile the extension's dev environment:
     • mvn compile
   • Assemble the extension:
     • mvn assembly:single
   • Clean the extension's run directory and unzip the target/rdf-transform-x.x.x.zip file in the extension's directory as documented in From Compiled Release above
     • rm -rf ~/.local/share/openrefine/extensions/rdf-transform*
     • unzip target/rdf-transform-$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout).zip -d ~/.local/share/openrefine/extensions

Long Steps

Sometimes you just have to do everything yourself. If you want or need to compile OpenRefine, then you'll probably want to create the jar file for RDF Transform to match. From the Short Steps, you'll notice these instructions have several inserted steps.

1. From some top level development directory, create a local repository for the RDF Transform extension:
   • At the top level development directory, make the OpenRefine Extensions directory--where you want the /rdf-transform sub-directory:
     • mkdir OpenRefineExtensions
   • Change directories to the OpenRefine Extensions directory:
     • cd OpenRefineExtensions
   • To create a new clone of the RDF Transform extension:
     • Clone the extension in the the OpenRefine Extensions directory--where you want the /rdf-transform sub-directory:
       • git clone https://github.com/AtesComp/rdf-transform
     • Change directories to the top level development directory:
       • cd ..
   • Alternatively, to update an existing clone, in the /rdf-transform directory:
     • Change directories to the RDF Transform development directory:
       • cd rdf-transform
     • Update the code:
       • git pull (or git fetch --all; git reset --hard; git pull for a forced refresh)
     • Change directories to the top level development directory:
       • cd ../..
2. Prepare the OpenRefine jar file:
   • Clone OpenRefine from the same top level development directory to create a local repository:
     • git clone https://github.com/OpenRefine/OpenRefine
   • Create the OpenRefine jars:
     • Change directories to OpenRefine:
       • cd OpenRefine
     • Clean OpenRefine's dev environment:
       • ./refine clean
     • Build OpenRefine:
       • ./refine build
     • Build the OpenRefine jar:
       • mvn package -DskipTests=true (builds the current version)
       • Among many other things, this builds the needed jar files:
         • openrefine-main.jar
         • openrefine-core.jar
         • openrefine-grel.jar
     • Change directories up one level:
       • cd ..
3. Process the OpenRefine jar files for the RDF Transform extension:
   • Change directories to the RDF Transform extension:
     • cd OpenRefineExtensions/rdf-transform
   • Adjust the pom.xml file to use the proper OpenRefine version ID
   • Install the OpenRefine jars in the Maven library for RDF Transform:
     • mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/main/webapp/WEB-INF/lib/openrefine-main.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
     • mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/modules/core/target/openrefine-core.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
     • mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=../../OpenRefine/modules/grel/target/openrefine-grel.jar -DcreateChecksum=true -DlocalRepositoryPath=./project-repository/
4. Compile the RDF Transform extension:
   • Clean the extension:
     • mvn clean
   • Update the extension's dependencies:
     • mvn dependency:resolve -U
   • Compile the extension:
     • mvn compile
   • Assemble the extension:
     • mvn assembly:single
   • Clean the extension's run directory and and unzip the target/rdf-transform-x.x.x.zip file in the extensions directory as documented in From Compiled Release above
     • rm -rf ~/.local/share/openrefine/extensions/rdf-transform*
     • unzip target/rdf-transform-$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout).zip -d ~/.local/share/openrefine/extensions

IDE Support:

There is a support section for IDEs in the code repository.

VSCode

For the VS Code IDE, there are support files to help the developer build and test the RDF Transform extension. Place these files in the project's .vscode top level directory.

• settings.json - includes settings for the Java build process
• task.json - various tasks for building and testing the extension in a development environment. They are currently based on a Linux OS (under other OSes, modify as needed).
  • The general purpose tasks are:
    • build clean assembly and update - Clean all in one: cleans existing Maven project, update Maven dependencies, compiles and assembles the project, and updates the extension directory for use by OpenRefine (OpenRefine should be restarted; do this for the initial build and when dependencies change). It combines:
      • clean, depend, compile, assemble, update ext
    • build assembly and update - (Default) All on one: compiles and assembles the Maven project and updates the extension directory for use by OpenRefine (OpenRefine should be restarted). It combines:
      • compile, assemble, update ext
    • update ext - Updates the user's OpenRefine extension directory with the assembled package for use by OpenRefine (OpenRefine should be restarted). It combines:
      • remove ext, copy ext
  • The specific tasks are (in order of proper implementation):
    • clean - Cleans the Maven project.
    • depend - Update the Maven project dependencies (see the pom.xml file).
    • compile - Compiles the Maven project.
    • assemble - Assembles the compiled Maven project into its zip file package.
    • remove ext - Removes the extension from the user's OpenRefine extension directory.
    • copy ext - Unzips the assembled package into the user's OpenRefine extension directory.

On Linux, the user's OpenRefine extension directory is ~/.local/share/openrefine/extensions.

In general, only the tasks build clean assembly and update and build assembly and update should be used. Use the other task independently as needed.