You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cookbook/localprocessing.rst
+98Lines changed: 98 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,3 +4,101 @@ Client-side (local) processing
4
4
5
5
.. warning::
6
6
This is a new experimental feature and API, subject to change.
7
+
8
+
Background
9
+
----------
10
+
11
+
The client-side processing functionality allows to test and use openEO with its processes locally, i.e. without any connection to an openEO back-end.
12
+
It relies on the projects `openeo-pg-parser-networkx <https://github.com/Open-EO/openeo-pg-parser-networkx>`_, which provides an openEO process graph parsing tool, and `openeo-processes-dask <https://github.com/Open-EO/openeo-processes-dask>`_, which provides an Xarray and Dask implementation of most openEO processes.
13
+
14
+
Installation
15
+
------------
16
+
17
+
.. note::
18
+
This feature requires ``Python>=3.9`` and has been tested
19
+
with ``openeo-pg-parser-networkx==2023.3.1`` and
20
+
``openeo-processes-dask==2023.3.2``
21
+
22
+
.. code:: bash
23
+
24
+
pip install openeo[localprocessing]
25
+
26
+
Usage
27
+
-----
28
+
29
+
Local Collections
30
+
~~~~~~~~~~~~~~~~~
31
+
32
+
Every openEO process graph relies on data, which was always provided by a cloud infrastructure (the openEO back-end) until now.
33
+
The client-side processing adds the possibility to read and use local netCDFs, geoTIFFs and ZARR files for your experiments.
34
+
35
+
If you want to use our sample data, please clone this repository:
With some sample data we can now check the STAC metadata for the local files by doing:
42
+
43
+
.. code:: python
44
+
45
+
from openeo.local import LocalConnection
46
+
local_data_folders = [
47
+
"./openeo-localprocessing-data/sample_netcdf",
48
+
"./openeo-localprocessing-data/sample_geotiff",
49
+
]
50
+
local_conn = LocalConnection(local_data_folders)
51
+
local_conn.list_collections()
52
+
53
+
This code will parse the metadata content of each netCDF, geoTIFF or ZARR file in the provided folders and return a JSON object containing the STAC representation of the metadata.
54
+
If this code is run in a Jupyter Notebook, the metadata will be rendered nicely.
55
+
56
+
57
+
.. tip::
58
+
The code expects local files to have a similar structure to the sample files provided `here <https://github.com/Open-EO/openeo-localprocessing-data.git>`_.
59
+
If the code can not handle you special netCDF, you can still modify the function that reads the metadata from it `here <https://github.com/Open-EO/openeo-python-client/blob/90c0505fae47c56746e49c91476be5147be6e1d0/openeo/local/collections.py#L19>`_ and the function that reads the data `here <https://github.com/Open-EO/openeo-python-client/blob/90c0505fae47c56746e49c91476be5147be6e1d0/openeo/local/processing.py#L26>`_.
60
+
61
+
Local Processing
62
+
~~~~~~~~~~~~~~~~
63
+
64
+
Let's start with the provided sample netCDF of Sentinel-2 data:
As you can see in the previous example, we are using a call to execute() which will execute locally the generated openEO process graph.
88
+
In this case, the process graph consist only in a single load_collection, which performs lazy loading of the data. With this first step you can check if the data is being read correctly by openEO.
89
+
90
+
Looking at the metadata of this netCDF sample, we can see that it contains the bands B04, B03, B02, B08 and SCL.
91
+
Additionally, we also see that it is composed by more than one element in time and that it covers the month of June 2022.
92
+
93
+
We can now do a simple processing for demo purposes, let's compute the median NDVI in time and visualize the result:
0 commit comments