Skip to content

Replace existing install script with a newer one that uses gem-based installation #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
christopherwharrop-noaa wants to merge 7 commits into
base: develop
Choose a base branch
from
Open

Replace existing install script with a newer one that uses gem-based installation #108

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
84775f3
Replace tarballs with gem installation
christopherwharrop-noaa Jul 22, 2024
57ce525
Update gem paths in command executables
christopherwharrop-noaa Jul 22, 2024
8532524
Trim install script and remove obsolete code
christopherwharrop-noaa Jul 22, 2024
9047431
Update command scripts to add lib paths to gems
christopherwharrop-noaa Jul 22, 2024
e5a47d0
Remove old INSTALL script and tarfiles
christopherwharrop-noaa Jul 22, 2024
87870fd
Include options for non-standard libxml2 installation locations (#116)
natalie-perlin May 15, 2025
7bd0a06
Fix incorrect comment
christopherwharrop-noaa May 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 47 additions & 158 deletions INSTALL
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,29 @@
#!/bin/bash
#!/usr/bin/env bash

# Script to install required external packages

# Get the OS
OS=$(uname -s)

# Set location of WFM
WFM_DIR=$(pwd)

# Set location where tarfiles for packages are kept
TAR_DIR=${WFM_DIR}/tarfiles
WFM_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

# Set location where packages are built
BUILD_DIR=${WFM_DIR}/build

# Set the location where libraries are installed
LIB_DIR=${WFM_DIR}/lib
# Set the location where gem dependencies are installed
GEM_DIR=${WFM_DIR}/gems

# Process arguments
for arg in $@; do
case $arg in
--with-ruby=*) RUBY=$(echo $arg | cut -f 2 -d "=")/bin/ruby;;
--with-xml2-dir=*) xml2dir=$(echo $arg | cut -f 2 -d "=");;
--with-xml2-include=*) xml2include=$(echo $arg | cut -f 2 -d "=");;
--with-xml2-config=*) xml2config=$(echo $arg | cut -f 2 -d "=");;
--with-xml2-lib=*) xml2lib=$(echo $arg | cut -f 2 -d "=");;
-h|--help) echo
echo "Run this script to install the Workflow Manager"
echo
echo "INSTALL [--with-ruby=/path/to/ruby]"
echo 'INSTALL [--with-ruby=/path/to/ruby]'
echo '[--with-xml2-dir=/path/to/${xml2-dir}]'
echo '[--with-xml2-include=/path/to/${xml2-dir}/include]'
echo '[--with-xml2-config=/path/to/xml2-config]'
echo '[--with-xml2-lib=/path/to/${xml2-dir}/lib]'
echo
exit;;
*)
Expand All @@ -48,13 +47,34 @@ else
exit 1
fi

GEM_ARGS=""
# Check whether libxml2 headers, libs, and xml2-config are defined
# xml2dir
if [[ -n "$xml2dir" && -d "$xml2dir" ]]; then
GEM_ARGS+=" --with-xml2-dir="$xml2dir
[[ -z "${xml2include+x}" || ! -d "${xml2include}" ]] && xml2include=${xml2dir}/include
[[ -z "${xml2lib+x}" || ! -d "${xml2lib}" ]] && xml2lib=${xml2dir}/lib
[[ -z "${xml2config+x}" || ! -d "${xml2config}" ]] && xml2config=${xml2dir}/bin/xml2-config
fi
# Add configuration options if they are set:
if [[ -n "${xml2include}" && -d "${xml2include}" ]]; then
GEM_ARGS+=" --with-xml2-include="$xml2include
fi

if [[ -n "${xml2lib}" && -d "${xml2lib}" ]]; then
GEM_ARGS+=" --with-xml2-lib="$xml2lib
fi

if [[ -n "${xml2config}" && -f "${xml2config}" ]]; then
GEM_ARGS+=" --with-xml2-config="$xml2config
fi
# Abort if any command returns an error
set -e

# Make sure ruby is at least 2.4.0 or higher
good_ruby=$($RUBY -e "puts RUBY_VERSION >= '2.4.0'")
# Make sure ruby is at least 3.0.0 or higher
good_ruby=$($RUBY -e "puts RUBY_VERSION >= '3.0.0'")
if [ "$good_ruby" = "false" ]; then
echo "Ruby 2.4.0 or higher is required"
echo "Ruby 3.0.0 or higher is required"
exit 1
fi

Expand All @@ -67,147 +87,16 @@ for file in $(ls sbin | grep -v '\.sh'); do
fi

# Replace first line with correct shebang
sed "1s:.*:#!${RUBY}:" sbin/$file > sbin/$file.new
mv sbin/$file.new sbin/$file

# Add path to libxml2 for AIX
if [ "$OS" = "AIX" ]; then
sed "2s%.*%ENV['LIBPATH']=\"${LIB_DIR}/libxml2/lib:#{ENV['LIBPATH']}\"%" sbin/$file > sbin/$file.new
mv sbin/$file.new sbin/$file
fi

# Make sure the new file is executable
chmod ugo+x sbin/$file
sed "1s:.*:#!${RUBY}:" -i sbin/$file

done

# Package versions
LIBXML2_VERSION="2.12.6"
LIBXML_RUBY_VERSION="5.0.3"
SYSTEMTIMER_VERSION="1.2.3"
SQLITE3_VERSION="autoconf-3450300"
SQLITE3_RUBY_VERSION="1.7.3"
OPEN4_VERSION="1.3.4"
RUBYSL_DATE_VERSION="2.0.9"
RUBYSL_PARSEDATE_VERSION="1.0.1"
THREAD_VERSION="0.2.2"

# Set AIX_CFLAGS and AIX_LDFLAGS
if [ "$OS" = "AIX" ]; then
AIX_CFLAGS="-maix64 -g0"
AIX_LDFLAGS="-maix64 -g0"
else
AIX_CFLAGS=""
AIX_LDFLAGS=""
fi

# Create the build directory
mkdir -p ${BUILD_DIR}

# Install rubysl-date
echo "==========================================="
echo "= INSTALLING rubysl-date"
echo "==========================================="
cd ${LIB_DIR}
gunzip -c ${TAR_DIR}/rubysl-date-${RUBYSL_DATE_VERSION}.tar.gz | tar -xvf -
rm -rf rubysl-date
mv rubysl-date-${RUBYSL_DATE_VERSION} rubysl-date

# Install rubysl-parsedate
echo "==========================================="
echo "= INSTALLING rubysl-parsedate"
echo "==========================================="
cd ${LIB_DIR}
gunzip -c ${TAR_DIR}/rubysl-parsedate-${RUBYSL_PARSEDATE_VERSION}.tar.gz | tar -xvf -
rm -rf rubysl-parsedate
mv rubysl-parsedate-${RUBYSL_PARSEDATE_VERSION} rubysl-parsedate

# Install Open4
echo "==========================================="
echo "= INSTALLING Open4"
echo "==========================================="
cd ${LIB_DIR}
gunzip -c ${TAR_DIR}/open4-${OPEN4_VERSION}.tar.gz | tar -xvf -
rm -rf open4
mv open4-${OPEN4_VERSION} open4

# Install libxml2
echo "==========================================="
echo "= INSTALLING libxml2 ="
echo "==========================================="
cd ${BUILD_DIR}
unxz -c ${TAR_DIR}/libxml2-${LIBXML2_VERSION}.tar.xz | tar -xvf -
cd libxml2-${LIBXML2_VERSION}
if [ "$LIBXML2_VERSION" = "2.9.0" ]; then # version 2.9.0 requires must be patched
echo "==========================================="
echo "= PATCHING libxml2 ="
echo "==========================================="
patch -p1 -i ${TAR_DIR}/libxml2-2.9.0.patch
fi
export OBJECT_MODE=64
./configure --prefix=${LIB_DIR}/libxml2 CFLAGS="$AIX_CFLAGS" LDFLAGS="$AIX_LDFLAGS" --without-python
make clean
make
rm -rf ${LIB_DIR}/libxml2
make install

# Install libxml-ruby
echo "==========================================="
echo "= INSTALLING libxml-ruby ="
echo "==========================================="
libxml2_opts="--with-xml2-lib=${LIB_DIR}/libxml2/lib --with-xml2-include=${LIB_DIR}/libxml2/include/libxml2"
cd ${BUILD_DIR}
gunzip -c ${TAR_DIR}/libxml-ruby-${LIBXML_RUBY_VERSION}.tar.gz | tar -xvf -
cd libxml-ruby-${LIBXML_RUBY_VERSION}/ext/libxml
echo ${RUBY} extconf.rb $libxml2_opts
${RUBY} extconf.rb $libxml2_opts
if [ "$OS" = "AIX" ]; then
:
else
sed -i "/^ldflags/ s:$: -Wl,-rpath,${LIB_DIR}/libxml2/lib:" Makefile
fi
make clean
make
rm -rf ${LIB_DIR}/libxml-ruby
mkdir ${LIB_DIR}/libxml-ruby
cp -r ${BUILD_DIR}/libxml-ruby-${LIBXML_RUBY_VERSION}/lib/* ${LIB_DIR}/libxml-ruby
cp libxml_ruby.so ${LIB_DIR}/libxml-ruby

# Install sqlite3
echo "==========================================="
echo "= INSTALLING sqlite3 ="
echo "==========================================="
cd ${BUILD_DIR}
unxz -c ${TAR_DIR}/sqlite-${SQLITE3_VERSION}.tar.xz | tar -xvf -
cd sqlite-${SQLITE3_VERSION}
./configure --prefix=${LIB_DIR}/sqlite3 CFLAGS="-Os -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_THREADSAFE=1 $AIX_CFLAGS $AIX_CFLAGS" LD_FLAGS="$AIX_LDFLAGS"
make clean
make
rm -rf ${LIB_DIR}/sqlite3
make install

# Install sqlite3-ruby
echo "==========================================="
echo "= INSTALLING sqlite3-ruby ="
echo "==========================================="
cd ${BUILD_DIR}
gunzip -c ${TAR_DIR}/sqlite3-ruby-${SQLITE3_RUBY_VERSION}.tar.gz | tar -xvf -
cd sqlite3-ruby-${SQLITE3_RUBY_VERSION}/ext/sqlite3
export LD_RUN_PATH=${LIB_DIR}/sqlite3/lib
${RUBY} extconf.rb -- --with-sqlite3-include=${LIB_DIR}/sqlite3/include --with-sqlite3-lib=${LIB_DIR}/sqlite3/lib --enable-system-libraries
make clean
make
rm -rf ${LIB_DIR}/sqlite3-ruby
mkdir ${LIB_DIR}/sqlite3-ruby
cp -r ${BUILD_DIR}/sqlite3-ruby-${SQLITE3_RUBY_VERSION}/lib/* ${LIB_DIR}/sqlite3-ruby
cp sqlite3_native.so ${LIB_DIR}/sqlite3-ruby/sqlite3
unset LDFLAGS

# Insteall thread
echo "==========================================="
echo "= INSTALLING Thread"
echo "==========================================="
cd ${LIB_DIR}
gunzip -c ${TAR_DIR}/thread-${THREAD_VERSION}.tar.gz | tar -xvf -
rm -rf thread
mv thread-${THREAD_VERSION} thread
# Install gem dependencies into local gem repository
gem install -i ${GEM_DIR} --no-user-install --no-document \
open4:1.3.4 \
rubysl-date:1.0.1 \
rubysl-parsedate:1.0.1 \
sqlite3:1.7.3 \
thread:0.2.2 \
libxml-ruby:5.0.3 \
-- $GEM_ARGS
2 changes: 1 addition & 1 deletion lib/workflowmgr/workflowdoc.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ class WorkflowXMLDoc

require 'time'

require 'libxml-ruby/libxml'
require 'libxml'
require 'workflowmgr/utilities'
require 'workflowmgr/cycledef'
require 'workflowmgr/workflowlog'
Expand Down
2 changes: 1 addition & 1 deletion lib/workflowmgr/workflowio.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ module WorkflowMgr
class WorkflowIO

require 'fileutils'
require 'libxml-ruby/libxml'
require 'libxml'


##########################################
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotoboot.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow status library
require 'workflowmgr/workflowengine'
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotocheck.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow status library
require 'wfmstat/statusengine'
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotocomplete.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow status library
require 'workflowmgr/workflowengine'
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotorewind.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow engine library
require 'workflowmgr/workflowengine'
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotorun.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow engine library
require 'workflowmgr/workflowengine'
Expand Down
13 changes: 5 additions & 8 deletions sbin/rocotoserver
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@
# Get the base directory of the WFM installation
__WFMDIR__=File.expand_path("../../",__FILE__)

# Add include paths for WFM and libxml-ruby libraries
# Add include paths for WFM and gem dependencies
$:.unshift("#{__WFMDIR__}/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-date/lib")
$:.unshift("#{__WFMDIR__}/lib/rubysl-parsedate/lib")
$:.unshift("#{__WFMDIR__}/lib/libxml-ruby")
$:.unshift("#{__WFMDIR__}/lib/sqlite3-ruby")
$:.unshift("#{__WFMDIR__}/lib/SystemTimer")
$:.unshift("#{__WFMDIR__}/lib/open4/lib")
$:.unshift("#{__WFMDIR__}/lib/thread/lib")
Dir["#{__WFMDIR__}/gems/gems/*"].each do |gemdir|
next if gemdir == "." or gemdir == ".."
$:.unshift("#{gemdir}/lib")
end

# Load workflow engine library
require 'tmpdir'
Expand Down
Loading