Stacks vs Tensors

Stacks

Stacks are defined as collections of channels that have been geometrically aligned. Each channel in a stack is stored separately as an ENVI file (with its corresponding .hdr header and .img data file), and every file in the collection maintains the same (highest) resolution, size, and georeference so that there is pixel-to-pixel correspondence across channels.
In essence, a stack is the very first processed state of Earth observation (EO) data, created by a satellite-specific Processor that organizes the input imagery into a folder of aligned channels.

Key attributes of stacks include:

Geometric Consistency: All channels have been rescaled and realigned into a common coordinate system, ensuring that the data across files are in perfect registration.
Ephemeral but Traceable: Although stacks are considered temporary data objects, they are fully traceable. This means that even if a stack is deleted, it can be re-instantiated to an identical state from the original raw data while retaining the same unique identifier.
Standardization: The stack represents an intermediary step where raw satellite data (from various sources) is standardized without any filtering, scaling, or irreversible modifications. This standardization is a precursor to all subsequent processing steps.
File Structure: Each channel is stored as a pair of files in ENVI format (.hdr and .img).

In order to produce a stack from the raw EO data, a variety of satellite-specific Processors are available. These Processors are invoked using CLI commands (typically in the form task make stack processor_name) and are designed to handle the specific formats and characteristics of the input data.

Available Processors of Optical data

Processor in OCLI	Input satellite data	Processing level	Radiometric correction	Geometric correction	Orthorectified	Atmospheric correction
s2	Sentinel-2	Level 2A and 2B	yes	yes	yes	yes
landsat	Landsat-8	Level 2	yes	yes	yes	yes
aster	ASTER	Level 1B	yes	yes	no	no
aster-l1t	ASTER	Level 1T	yes	yes	yes	no
wv3	WorldView-3	Level 2	yes	yes	no	no

Sentinel-2 and Landsat-8 Level 2 data are atmospherically corrected and no further pre-processing of this data is necessary, but ASTER and WorldView-3 data are not, meaning they are still Top Of Atmosphere, so if Bottom of Atmosphere is needed for analysis, it should be implemented before the next step of Assembling Tensors.

In open source OCLI code, an example of Sentinel-2 processors is in ocli/processor/s2_processor as a tool that can be used by many AIKPs.

Available Processors of SAR data

Processor in OCLI	Input satellite data	Processing level
sarpy	Sentinel-1	SLC
sarpy	TSX/PAZ	SLC

PRO OCLI has implemented a Stack Processor with extended capabilities.
Input satellite images in this case are SLC (Single Look Complex) SAR data that contain both amplitude and phase information, preserved in a complex format. This data is considered primary and unaltered (except for basic corrections) and serves as the starting point for more advanced processing.

Output of sarpy Processor, in example of Sentinel-1, is:

Sigma0 = normalized radar backscatter coefficient derived from SLC SAR data that indicates how much of the radar signal is returned (backscattered) to the sensor from a target surface. In the case of Sentinel-1, they are sigma_vv and sigma_vh.
Coherence = measure of the similarity or correlation between two SAR images acquired for different time periods. These Stack channels are only calculated if the AIKP uses both master and slave SAR image, and, for Sentinel-1, are coherence_vv and coherence_vh.
Tilt = simply labeled as tilt provides information related to angular variations in the imaging geometry and is derived from the Digital Elevation Model (DEM) as a transformed function of angle between surface normal and local vertical direction.

Tensors

Tensors represent the next step of processed data and involve more advanced preprocessing to prepare the data for analysis.
While a Stack is essentially a standardization of the raw input, Tensors are created from a Stack and saved as a NumPy ndarray.
Tensors are often also referred to as ARD “data hypercubes” because they encapsulate a multidimensional (multichannel) dataset that is fixed in time.

Enhanced Processing: Before being converted into a tensor, the data undergoes satellite-specific preprocessing such as normalization, filtration, and denoising.

Available Filters

During the tensor assembly process, various filters can be applied to enhance the data quality. The OCLI system provides a set of filters in the ocli.ai.filter module:

anisotropic_diffusion: An edge-preserving diffusion filter that smooths noise while preserving important edges in the image. This filter has been moved from the PRO version to the standard version, making it available to all users.
bm3d: Block-Matching and 3D filtering, an advanced denoising algorithm that groups similar 2D image fragments into 3D data arrays. (PRO version)
nlmd: Non-Local Means Denoising, which replaces the value of a pixel with an average of similar pixels in the image.
sarblf: A specialized filter for SAR (Synthetic Aperture Radar) data. (PRO version)
gabor: A filter that can be used for texture analysis and feature extraction (available in PRO version).
bm3d_with_post: An enhanced version of the bm3d filter with post-processing (available in PRO version).

These filters can be configured and applied in the assemble_kernel function of an AIKP’s assemble_recipe.py file using the Filter class. For example:

from ocli.ai.filter import Filter

# Create a filter instance
filter_instance = Filter('anisotropic_diffusion', {'gamma': 0.2, 'option': 3})

# Apply the filter to an image
filtered_image = filter_instance.apply(image)

The filter configuration can also be controlled through task configuration parameters, which can be set using the update_filter_config function.

Multidimensional Representation: All the channels are combined and stored in a single multidimensional float-valued NumPy array (typically saved in a .npy file). This setup effectively consolidates all the channels into one data object.
Accompanying Metadata and Mask: Along with the multidimensional array, there is a secondary 2D binary NumPy array which acts as an alpha mask. Additionally, metadata is stored in accompanying files (e.g., .npy.hdr for georeferencing information and .npy.json for other channel-related metadata).
Frozen State: Once created, tensors become “frozen” in time, representing a stable and final state before any further high-level analytical or computational tasks.
Creation Process: An AIKP-specific assemble kernel (in assemble_recipe.py) invokes this conversion from a stack to a tensor.

In OCLI code

The template.py defines get_assembler method that instantiates Assembler object:

@classmethod
def get_assembler(cls, recipe_py):
from ocli.ai.assemble import Assemble
return Assemble(recipe_py)

and is triggered by CLI command ai basic assemble full/zone

This procedure instantiates an Assembler implementation from ocli/ai/assemble and initialises the object with the task’s recipe (that has previously been prepared with task make recipe CLI command).
The Assembler object uses of the key-values, defined in the task’s recipe, in particular OUTDIR, zone, products, stack_path, channels.
Note that the recipe_py here is not the task’s recipe, but rather a python file assemble_recipe.py in the AIKP directory that provides an assemble_kernel function.

The example of commented assemble_recipe.py can be found in ocli/aikp/mondrian/s2/assemble_recipe.py and it has some mandatory elements:

input_list = [
{"fname": "..", …},
{"fname": "..", …},
…]

This is a list of inputs from a previously created Stack wherefname has to correspond to the name of respective stack ENVI files (.img/.hdr).
Other fields are optional and are used as tags in assemble_kernel for selection of the requested subset of channels.

tnsr_fnames = {
'full': {'tnsr': 'full_tnsr.npy', 'bad': 'full_bd.npy', 'hdr': 'full_tnsr.npy.hdr', 'meta':
'full_tnsr.json'},
'zone': {'tnsr': 'zone_tnsr.npy', 'bad': 'zone_bd.npy', 'hdr': 'zone_tnsr.npy.hdr', 'meta':
'zone_tnsr.json'}}

This describes filenames for the outputs for two cases - when either zone-cropped (ROI-cropped) or full-size tensor is assembled.

def assemble_kernel(inputs: LayerSelector, outputs: LayerSelector, bad_pixels):

This is where AIKP developer defines all the preprocessing computations that happen in the process of assembling the Tensor.

Inputs are given to the kernel in the form of a LayerSelector object taken from input_list.
The LayerSelector provides a function to select a subset of the inputs by taking requested tag-value and returning only the channels for which those tag-values match.
For example, it is possible to select only coherence channels by requesting inputs.get(type='coh') or only master channels with inputs.get(ms='mst').
The get function returns a list of channels that can be iterated over or a channel at a specific index may be selected. If the goal is to select only one channel, then the 0-element of the list has to be extracted: inputs.get(type='coh', pol='vv')[0].

Outputs are also in the form of a LayerSelector object, but this time the AIKP developer is responsible for populating it with the data resulting from processing.

The population is achieved by multiple calls of append method like this:
outputs.append(img, name='example-name') The first argument would be the processed array, followed by a list of tag-value pairs, from which tag name is mandatory, as it describes the channel name in the tensor.
bad_pixels also provides an append method, but this does not require/allow tags and only takes a binary mask of pixels that the processing algorithm decides to mark as not suitable for further processing.
The bad_pixel masks are combined for all the channels with OR logic.
The resulting mask is stored in a separate .npy file.
Georeferencing information is preserved in corresponding npy.hdr header (formatted according to ENVI standard)
Output tags are saved in corresponding .npy.json.

All these files are saved in the task’s AI_RESULTS directory.