ocli.aikp.ts_mediator (Batch processing)

This is an example/boilerpate AIKP where you can get the inspiration on how to implement the batch processing.

A minimal AIKP that reuses existing CREODIAS finders to fetch satellite products (Sentinel‑1/2, Landsat‑8) and stores the results in the project cache. It prefers invoking/extending built‑ins over re‑implementing logic.

First of all you need to get a list of products (please see the ocli/aikp/ts_mediator/cli.py command task template products). Then batch processing should consist of the 3 states: “Initialization” -> “Processing Loop” -> “Merge results”.

  1. “Initialization” – Setup filters and all parameters required. Get the product list. Store it in a task. Create ocli sub-environment with sum-task.

  2. “Processing Loop” – Run the sub-task which performs the specific analysis (ex. 1 task run 1 time layer).

  3. “Merge results” – Merge results from sub-task run into publication ready product.

task template products or How to get filtered products list in one AIKP

+----------------------+           +-----------------------------+
| Task (ts_mediator)   |           |  Finder Dispatch            |
|  - sat_family_name   |  selects  |  finder_by_task(task)       |
|  - finder.* filters  +---------->+  (s1/s2/landsat_8)->        |
|  - ROI               |           |  FinderCreodias* class      |
+----------+-----------+           +--------------+--------------+
           |                                      |
           | inject_finder_keys()                 |
           |                                      v
           |                          +-----------------------------+
           |                          | CREODIAS Finder (RESTO API) |
           |                          |  - builds query             |
           |                          |  - requests search.json     |
           |                          |  - parses features -> GDF   |
           |                          +-----------------------------+
           |                                      |
           |                            save_meta_to_cache(.gpkg)
           v                                      v
+----------------------+              +-----------------------------+
| CLI (ts_mediator)    |              | Project .cache (GeoPackage) |
|  task template       |              |  - reused by global product |
|   products           |              |    commands                 |
+----------------------+              +-----------------------------+
                                                   |
                                                   v
                                        product list/show/load (global)

Key points:

  • Uses sat_family_name to choose the correct CREODIAS finder.

  • Uses ROI from the task to constrain queries.

  • Stores results in project cache (.cache/finder_*.gpkg).

  • finder.* keys live in the task config and override defaults.

CLI Usage

Prerequisites:

  • An active project with ROIs set up.

  1. Create a task from this template

  • task create --template ocli.aikp.ts_mediator --roi <roi_name> --name ts_demo --activate

  1. Select a satellite family (optional)

  • Sentinel‑2: task set sat_family_name=s2

  • Sentinel‑1: task set sat_family_name=s1

  • Landsat‑8: task set sat_family_name=landsat_8

  1. Set finder filters in task config (optional)

  • Date range: task set finder.startDate=2025-01-01

  • Date range: task set finder.completionDate=2025-02-01

  • Cloud cover: task set finder.cloudCover="[0,20]"

  1. Fetch and cache products with the AIKP command

  • task template products --reload --max-items 200 --columns "title,completionDate,cloud_cover"

  • What happens:

    • Uses finder.* overrides + task ROI

    • Calls CREODIAS REST API and writes .gpkg cache

    • Prints a quick preview table

  1. Browse results using global product commands (read from cache)

  • product list --column productId --column startDate --limit 20

  • product show <PRODUCT_ID>

  • product load can also perform the fetch-and-cache step; ts_mediator just gives you a lighter, template‑local variant via products.

Notes

  • Changing sat_family_name switches finder class automatically.

  • The cache is per project: PROJECT/.cache/finder_*.gpkg.

  • You can refine filters at any time via task set finder.<key>=<value> and rerun products.

  • Use --no-reload to reuse cached results.

Troubleshooting

  • Empty list: verify ROI geometry exists in the task, date range is valid, and API is reachable.

  • Unknown column in --columns: omit it or check columns via product list --columns.

  • Authentication isn’t required for public CREODIAS queries used here; corporate proxies/firewalls may still affect requests.

task template sandbox or How to create ocli sub-environment inside of current task folder

Please see the ocli/aikp/ts_mediator/cli.py command task template sandbox.

Idea is to invoke ocli commands in nested environment (isolated from current user’s environment). Basically we can set path to .tsarrc config via --home parameter.

To achieve it we next to:

  1. Create folders for sub-ocli (see “Sandbox path” and “Sandbox Projects Home”).

  2. Generate appropriate configration file (see “Sub-ocli configuration file”).

  3. Create/Initialize sub-task (see “Create sub-task”).

Then using method like invoke_sub_ocli (see the ocli/aikp/ts_mediator/cli.py) in the initialized sub-ocli environment you can run any predefined sequence of commands and have access to their results.

To save the execution state - basic state machine should be implemented.