Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor docs site structure #69

Merged
merged 1 commit into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ venv
dist
.idea
.vscode
.pre-commit-config.yaml
.pre-commit-config.yaml
.local
22 changes: 0 additions & 22 deletions docs/design/common-library/design.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Harvester
# Design

The harvester is implemented as workflows in the Flowable BPMN platform. Flowable Platform is a flexible framework for workflow and process automation. Its core is a native Business Process Modelling Notation (BPMN) 2.0 process engine that runs inside the Java Virtual Machine. Workflows need to be defined with the BPMN specification that supports parallel workflow steps, branching, error handling, etc. A workflow step can be set to type “external” to allow an individual worker to fetch tasks for a specific workflow step. This worker connects to the REST API, fetches tasks, and returns the status (e.g., completed, failed, error). Failed workflow steps can be automatically retried from the Flowable process engine. Errors can be modelled in the BPMN diagram, e.g., asking an operator how to proceed as a manual user task.

Expand Down
4 changes: 0 additions & 4 deletions docs/design/harvester/api/usage.md

This file was deleted.

3 changes: 0 additions & 3 deletions docs/design/more-design.md

This file was deleted.

3 changes: 0 additions & 3 deletions docs/getting-started/development-guide.md

This file was deleted.

21 changes: 20 additions & 1 deletion docs/getting-started/quick-start.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
# Quick Start

Quick start instructions - including installation, e.g. of a local instance.
To try out the Resource Registration building block you can install it locally or deploy the full Kubernetes setup as described in the [EOEPCA Deployment Guide](https://deployment-guide.docs.eoepca.org/current/building-blocks/resource-registration/).

## Registration API

To setup a local copy of the Registration API follow the instructions provided [here](https://github.com/EOEPCA/registration-api?tab=readme-ov-file#getting-started). This component is build upon the [pygeoapi](https://pygeoapi.io/) project to offer the [OGC API - Processes](https://ogcapi.ogc.org/processes/) to the user. An introduction to the OGC API Processes can be found in the Registration API [User Guide](../../user-guide/registration-api-usage/).

## Harvester

You can setup a local copy of the Harvester component using Docker and Docker Compose:

1. Run the Flowable workflow engine on your machine as described in the [Flowable Docker documentation](https://github.com/flowable/flowable-engine/tree/main/docker).
3. Clone the [Harvester Github repository](https://github.com/EOEPCA/registration-harvester.git).
4. Deploy the BPMN workflow definitions contained in the `workflows` directory on your local Flowable instance.
5. The `docker-compose.yml` in the project root directory defines the worker processes for the workflows. Adapt the workflow-specific configuration files to your enviroment and start everything with `docker compose up`.

!!! info "USGS M2M user account"
The Landsat workflow searches the [Landsat STAC API](https://landsatlook.usgs.gov/stac-server) and downloads the data from the [USGS Machine-to-Machine (M2M) API](https://m2m.cr.usgs.gov/). To access this system, a M2M user accout is required which can be created [here](https://ers.cr.usgs.gov/register). The credentials must be passed as the environment variables `M2M_USER` and `M2M_PASSWORD` to the worker processes.

!!! info "CDSE user account"
The Sentinel workflow searches and downloads the data from the [CDSE OData API](https://datahub.creodias.eu/odata/v1). To access this API, a CDSE user accout is required which can be created [here](https://identity.dataspace.copernicus.eu/auth/realms/CDSE/login-actions/registration?client_id=cdse-public&tab_id=0kmNL363Fs4). The credentials must be passed as the environment variables `CDSE_USER` and `CDSE_PASSWORD` to the worker processes.
Binary file modified docs/img/resource-registration-components.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@ mkdocs-autorefs==1.0.1
mkdocs-publisher==1.2.0
mkdocs-glightbox==0.4.0
mkdocs-swagger-ui-tag
#mkdocstrings[python]
26 changes: 26 additions & 0 deletions docs/user-guide/harvester-usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Harvester - User Guide

## Core Concepts

### Workflow engine

### Workflow definition

### Workflow instance

### Worker

## Built-in workflows

### Landsat

### Sentinel

## Developing workflows

This section descibes how to create your own workflow und run it with the Harvester.

### Design
### Submitting a workflow definition
### Executing a workflow

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Usage
# Registration API - User Guide

## Introduction

Expand Down
67 changes: 14 additions & 53 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,64 +7,43 @@ nav:
- EOEPCA Documentation: https://eoepca.readthedocs.io/
- Resource Registration:
- index.md
- Getting Started:
- getting-started/quick-start.md
- Design:
- Architecture: design/overview.md
- Components:
- Registration API (pygeoapi):
- Design: design/registration-api/design.md
- API:
- design/registration-api/api/endpoint-specification.md
- design/registration-api/api/usage.md
- Harvester:
- Getting Started:
- getting-started/quick-start.md
- getting-started/development-guide.md
- Design: design/harvester/design.md
- API:
- design/harvester/api/endpoint-specification.md
- design/harvester/api/usage.md
- eodm: https://geopython.github.io/eodm/
# - Common Library:
# - Design: design/common-library/design.md
# - API:
# - Reference:
# - Base: design/common-library/api/base.md
# - Datasets: design/common-library/api/datasets.md
# - Providers: design/common-library/api/providers.md
# - Resources: design/common-library/api/resources.md
# - design/common-library/api/usage.md
# - design/more-design.md
# - Usage:
# - usage/tutorials.md
# - usage/howtos.md
- Registration API: design/registration-api-design.md
- Harvester: design/harvester-design.md
# - eodm: https://geopython.github.io/eodm/
- User Guide:
- Registration API: user-guide/registration-api-usage.md
- Harvester: user-guide/harvester-usage.md
# - Administration:
# - admin/configuration.md
# - admin/maintenance.md
# - API:
# - api/endpoint-specification.md
# - api/usage.md
- API:
- Registration API: api-specification/registration-api-specification.md
- Harvester: api-specification/harvester-api-specification.md

theme:
# name: mkdocs
# name: readthedocs
name: material # https://squidfunk.github.io/mkdocs-material/creating-your-site/
navigation_depth: 4

# For `material` theme...
logo: img/favicon.ico
favicon: img/favicon.ico
features:
- navigation.footer
# - navigation.footer
- navigation.instant
- navigation.tracking
- navigation.tabs
- navigation.tabs.sticky
- navigation.sections
# - navigation.expand
# - toc.integrate
- navigation.top
- content.code.copy
- content.action.edit
- search.highlight
- toc.follow

extra_css:
- css/eoepca.css
Expand All @@ -74,24 +53,6 @@ plugins:
- glightbox
- search
- swagger-ui-tag
# - mkdocstrings:
# default_handler: python
# handlers:
# python:
# options:
# show_if_no_docstring: false
# filters:
# - "!checksum_funcs"
# - "!asset_changes"
# - "!folder_structure"
# - "!__log" # zu streng;
# - "!scene_id_pattern"
# - "!scene_id_pattern"
# - "!sensor_name"
# - "!usgs_path_structure"
# - "!checksum_settings"
# - "!variable_mappings"
# - "!BASE_QUERY"
# - pub-debugger:
# console_log:
# enabled: true
Expand Down