Custom AIKP development flow: Step-by-step guide

Extremely simple custom flow AIKP that reads user inputted GeoTIFF, finds domain minimum and domain maximum value, and saves them to a .txt file in AI_RESULTS directory of the user

Initialize (Create) environment

Initialization stage is an environment specific. Please select the desired one. When you are going to deal with all steps - please go to “Developing” stage.

Local Environment (Local docker)

Prerequisites for a local environment is: GIT, Docker and any IDE (or editor).

  1. Open your local terminal.

  2. Clone the ocli open source code git clone git@github.com:Opt-OSS/GoldenRAM.git ocli and go to cloned folder cd ocli.

  3. Open this source folder in your favourite editor or IDE.

  4. Copy ocli/aikp/boilerplate AIKP to a new folder ocli/aikp/working_demo. (Of course you can replace working_demo by another name).

  5. Now you have 5 files there

Sandbox environment (DOCLI)

  1. Open SSH connection

  2. Navigate to /home/user_name/OCLI/AIKP

  3. Make a directory with the name working_demo

  4. Copy paste 5 files from /home/user_name/AIKP/ocli_aikp_examples/ocli/boilerplate to /home/user_name/OCLI/AIKP/working_demo

  5. Now you have 5 files there

Developing

__init__.py = Marks the AIKP directory as a python package and defines Template class inherited from TemplateBoilerplate
config.py = AIKP-specific configuration defaults for tasks and recipes. This includes default paths and parameters used across the AIKP.
template.py = Task Template class (TemplateBoilerplate) a.k.a task controller which implements common task operations: validate task, create task, upgrade task, update recipe, ai path resolve, etc.
cli.py = Defines all custom AIKP specific CLI commands
recipe_schema.json = JSON Schema for recipe validation

  1. config.py

  • comment out line 73 custom_parameter - this demo only needs custom_input_path as a key
    TASK_DEFAULTS is what will be visible in the workflow when the AIKP is executed (‘task show’) and can be adjusted with end-user input (‘task set key=value’)

  1. template.py = basic “tools” are already here and can be edited and adjusted based on your need

  • Rename TemplateBoilerplate to TemplateWorkingDemo at line 23 (if done in step 6)

  • update_recipe classmethod
    Line 53: comment out 'custom_parameter': task.config['custom_parameter'],
    Non-necessary changes for better understanding and final output:
    Line 27: REQUIRED custom message
    Line 38: custom tag
    Line 49: custom visible metadata shown in UI after publication

  • validate_task classmethod
    Validation for input already here
    Error message can/should be more informative “Should be GeoTIFF file”
    Comment out lines 87-89 (about custom_parameter check)= do not need it in this demo

  • validate_schema classmethod
    Line 100: change ocli.aikp.boilerplate to ocli.aikp.working_demo

  • cli_task_mount classmethod
    Come back later to it

  1. __init__.py

  • rename TemplateBoilerplate in line 4 and 7 to TemplateWorkingDemo
    not necessary step but good for organisation and clear naming

  1. cli.py = place for custom commands and workflow one command at a time

  • comment out (or delete) 12-to the end

  • add to the beginning

from osgeo import gdal 
from ocli.core.exception import OCLIException 
from ocli.core import output

  • make a new custom command find_domain

@click.command('find_domain')
@pass_task
@ensure_task_resolved
def find_domain(task: Task):
    recipe = Recipe(resolve_recipe(task))
    # get user-inputted GeoTiFF from the path
    custom_input_path = task.config['custom_input_path']
    custom_input_path = Path(custom_input_path).expanduser()
    print(f"custom_input_path={custom_input_path.absolute()}!")
    # Open the GeoTIFF file
    dataset = gdal.Open(str(custom_input_path))
    if dataset is None:
        raise OCLIException(f"Failed to open {custom_input_path}")
    # Get the first band
    band = dataset.GetRasterBand(1)
    if band is None:
        raise OCLIException(f"No bands found in {custom_input_path}")
    # Compute min and max values
    stats = band.ComputeStatistics(False)
    if stats is None:
        raise OCLIException(f"Could not compute statistics for {custom_input_path}")
    domain_min, domain_max = stats[0], stats[1]
    output.info(f'Domain_min={domain_min}; domain_max={domain_max}')
    # Save to file
    output_dir = Path(recipe['OUTDIR'])
    output_dir.mkdir(parents=True, exist_ok=True)
    output_file = output_dir / "domain_values.txt"
    with open(output_file, "w") as f:
        f.write(f"Domain_min={domain_min}\nDomain_max={domain_max}\n")
    print(f"Saved domain values to {output_file}")
    task.save()

  1. Back to template.py

  • cli_task_mount classmethod
    Line 150: change command1 to find_domain
    Line 153: change command1 to find_domain

Test it

Prerequisites

rsync demo.tiff and demo_roi.geojson to /home/user_name/OCLI/USER_UPLOADS

Local Environment (Local docker)

Please follow the manual to build required images.

Make sure that the folder with test files are mounted to docker container.

Move them (files) it into $HOME/OCLI_HOME (where $HOME is your home folder). It will be mounted to the /root/OCLI_HOME (inside of container).

Run the ocli docker (from the root folder where the ocli course has been cloned).

OCLI_HOME=${OCLI_HOME:-"$HOME/OCLI_HOME"}
SSH_CREDENTIALS=${SSH_CREDENTIALS:-"$HOME/.ssh"}
OCLI_SOURCE=${OCLI_SOURCE:-"./"}

docker run -it \
  -v $OCLI_HOME:/root/OCLI_HOME \
  -v $SSH_CREDENTIALS:/root/.ssh \
  -v $OCLI_SOURCE:/root/ocli_install/ocli \
  ocli:latest \
  bash

Now you are inside ocli docker container. Install ocli as an editable package.

pip install -e ./ocli

Then start OCLI REPL.

ocli repl

CLI workflow

  1. create –name demo –activate

  2. list

  3. roi add –name demo /root/OCLI_HOME/USER_UPLOADS/demo_roi.geojson

  4. roi list

  5. task create –template ocli.aikp.working_demo –roi demo -n demo –activate

  6. Optional: task set friendly_name=”Demo/example/”

  7. task set custom_input_path=/root/OCLI_HOME/USER_UPLOADS/demo.tiff

  8. task make recipe

  9. task template find_domain

exit - from ocli repl, but you still in the ocli docker container.

  1. Go to /root/OCLI_HOME/AI_RESULTS/demo_demo and find the created .txt file

  2. Optional: Can drag-n-drop demo.tiff in QGIS to verify values are correct