Plugin : ckanext archiver

!ATTENTION! This plugins installer is experimental!

Installation

To install ckanext-archiver:

1. Activate your CKAN virtual environment:

easyckan exec

2. Install the ckanext-archiver and ckanext-report Python packages into your virtual environment:

pip install -e git+http://github.com/datagovuk/ckanext-report.git#egg=ckanext-report
pip install -e git+http://github.com/ckan/ckanext-archiver.git#egg=ckanext-archiver

3. Install the archiver dependencies:

pip install -r /usr/lib/ckan/default/src/ckanext-archiver/requirements.txt
pip install redis==2.10.1
pip install celery==2.5.5

4. Now create the database tables:

paster --plugin=ckanext-report report initdb --config=/etc/ckan/default/development.ini
paster --plugin=ckanext-archiver archiver init --config=/etc/ckan/default/development.ini

5. Enable plugins "archiver" and "report"

Add archiver report to the ckan.plugins setting in your CKAN config file.

easyckan exec nano /etc/ckan/default/development.ini

6. Add Celery configuration

Edit development.ini and paste the follow in content below the [DEFAULT] line.

easyckan exec nano /etc/ckan/default/development.ini

[app:celery]
BROKER_BACKEND = redis
BROKER_HOST = redis://ckan-supervisor/0
BROKER_URL = redis://ckan-supervisor/0
CELERY_RESULT_BACKEND = redis
CELERY_REDIS_HOST = ckan-supervisor
CELERY_REDIS_PORT = 6379
CELERY_REDIS_DB = 0
CELERY_REDIS_CONNECT_RETRY = True

7. Add Archiver configuration

Edit development.ini and paste the follow in content below the ckan.plugins = [...] line.

easyckan exec nano /etc/ckan/default/development.ini

# ckanext-archive
ckanext-archiver.archive_dir = /var/lib/ckan/archiver
ckanext-archiver.cache_url_root = http://localhost:8082

8. Add Apache HTTP Server

easyckan exec nano /etc/ckan/easyckan/plugins/archiver.sh

#!/bin/bash

# Reset permissions
mkdir -p /var/lib/ckan/archiver
touch /var/lib/ckan/archiver/nothing
chown -R 5000.2 /var/lib/ckan/archiver
chmod -R 776 /var/lib/ckan/archiver

# Start apache server
if ! [[ $(docker container ls | grep -i 'ckan-archiver') ]]; then
  docker rm -f ckan-archiver 2> /dev/null
  docker run -d --net=easyckan --name ckan-archiver -p 8082:80 \
       -v "/var/lib/ckan/archiver":/usr/local/apache2/htdocs/ \
       httpd:2.4-alpine
fi

9. Add Supervisor config file

Supervisor files should be placed in /etc/ckan/default/supervisor.

Redis is a key-value database with content keeped in memory. Is used by ckanext-archiver throw Celery.

easyckan exec nano /etc/ckan/easyckan/plugins/redis.supervisor

[program:redis]
command=redis-server

easyckan exec nano /etc/ckan/easyckan/plugins/archiver.supervisor

[program:archiver]
command=/usr/lib/ckan/default/bin/paster --plugin=ckanext-archiver celeryd2 run all -c /etc/ckan/default/development.ini
user=ckan
numprocs=1
stdout_logfile=/tmp/ckanext-archiver.log
stderr_logfile=/tmp/ckanext-archiver.log
autostart=true
autorestart=true
startsecs=10

[program:report]
command=bash -c ". /usr/lib/ckan/default/bin/activate && while true; do sleep 600 ; /usr/lib/ckan/default/bin/paster --plugin=ckanext-report report generate --config=/etc/ckan/default/development.ini ; done"
user=ckan
numprocs=1
stdout_logfile=/tmp/ckanext-report.log
stderr_logfile=/tmp/ckanext-report.log
autostart=true
autorestart=true
startsecs=10

10. Restart EasyCKAN containers

easyckan repair