openghg_inversions.inversion_data.preparation#

Shared mechanics and durable data contracts for inversion preparation.

prepare_rhime_inputs returns backend-neutral observations, sensitivities, basis metadata, and site metadata; component-specific model arrays are intentionally absent. The temporary legacy fixed-basis orchestration is owned by openghg_inversions.hbmcmc.preparation and composes the lower-level retrieval, filtering, basis, and array helpers retained here.

RhimePreparedInputs validates the relationships between these labeled arrays when it is constructed. When the retained basis-functions object provides validated(), preparation uses the returned copy after that method has rechecked its mutable flux, operator data, and source labels. Compatible objects without that hook are retained unchanged.

Preparation can read OpenGHG object stores or local merged-data artifacts, write merged-data and basis artifacts, emit warnings and progress messages, and record timing information. These backend-neutral preparation functions do not construct a PyMC model.

class openghg_inversions.inversion_data.preparation.RhimeMergedData(fp_all: dict, site_options: _SiteOptions)#

Bases: object

Merged RHIME data and complete site-aligned metadata between stages.

Parameters:

  • fp_all – Merged per-site datasets plus shared flux, boundary-condition, and calibration entries.

  • site_options – Complete site-aligned acquisition options retained after retrieval or filtering.

Notes

This is a supported orchestration handoff. Its datasets remain backend-neutral and may be Dask-backed; later stages must treat them as borrowed.

property averaging_period: tuple[str | None, ...]#

Retained averaging periods aligned to sites.

fp_all: dict#
property platform: tuple[str | None, ...]#

Retained observation platforms aligned to sites.

site_options: _SiteOptions#
property sites: tuple[str, ...]#

Retained site names.

class openghg_inversions.inversion_data.preparation.RhimePreparedInputs(inv_inputs: Dataset, basis_functions: FluxWeightedBasis, site_metadata: Dataset)#

Bases: object

Modern RHIME preparation and durable serialization contract.

Site labels and site-aligned metadata are owned by site_metadata. Integer inv_inputs.site_indicator values are derived as zero-based project-schema positions into its site coordinate; they are distinct from CF compression-by-gathering indexes. The invariant is site_metadata.site[site_indicator] == nmeasure.site. site_names remains available in memory for existing model code, but is regenerated from site_metadata rather than serialized.

Parameters:

  • inv_inputs – Canonical inversion inputs consumed by RHIME model builders.

  • basis_functions – Retained flux basis object used to derive output-boundary basis and flux arrays.

  • site_metadata – Dataset indexed by the authoritative site coordinate. Every variable contains exactly one value per site, and averaging_period is required. Observation metadata that is genuinely constant per site may also be stored here. Values that vary within a site, such as satellite or aircraft release locations, must instead remain observation-aligned arrays. Such arrays may be carried alongside the inversion arrays without implying that model builders consume them.

Raises:

ValueError – If site metadata, measurement indexing, or multi-source labels are inconsistent.

property averaging_period: tuple[str | None, ...]#

Return averaging periods aligned to sites.

property basis_artifact_path: str | None#

Return the provenance-only basis artifact path.

property basis_artifact_source: str#

Return retained basis provenance, defaulting to generated.

basis_functions: FluxWeightedBasis#
classmethod from_datatree(dt: DataTree) → Self#

Construct prepared RHIME inputs from a version-1 DataTree.

Parameters:

dt – DataTree using the openghg_inversions.rhime_prepared_inputs schema.

Returns:

Reconstructed prepared inputs with the canonical inversion-input MultiIndexes and embedded basis object restored, including retained multisource basis ordering.

Raises:

  • ValueError – If the prepared schema, site metadata, serialized MultiIndex, site indicators, or source labels are malformed.

  • KeyError – If a required child node is missing.

classmethod from_legacy_inputs(inv_inputs: Dataset, basis_functions: FluxWeightedBasis, sites: Sequence[str], averaging_period: Sequence[str | None], basis_artifact_source: str | None = None, basis_artifact_path: str | None = None, site_lats: Sequence[float] | None = None, site_lons: Sequence[float] | None = None) → Self#

Adapt the former positional fields to the labeled-data contract.

Parameters:

  • inv_inputs – Canonical inversion inputs.

  • basis_functions – Retained basis object.

  • sites – Site labels in indicator-decoding order.

  • averaging_period – Observation periods aligned to sites.

  • basis_artifact_source – Optional basis provenance value.

  • basis_artifact_path – Optional basis provenance path.

  • site_lats – Optional legacy release latitudes, one per site.

  • site_lons – Optional legacy release longitudes, one per site.

Returns:

Prepared inputs using labeled site metadata.

inv_inputs: Dataset#
classmethod load(file_path: str | Path) → Self#

Load prepared RHIME inputs from a NetCDF or Zarr artifact.

Parameters:

file_path – Prepared-input artifact previously written by save.

Returns:

Fully loaded prepared RHIME inputs with no open file handles.

Raises:

  • OSError – If the artifact cannot be opened.

  • RuntimeError – If all available storage backends fail at runtime.

  • ValueError – If the artifact schema or metadata is invalid.

  • KeyError – If a required child node is missing.

save(output_file: str | Path, output_format: Literal['netcdf', 'zarr'] | None = None) → None#

Save prepared RHIME inputs to NetCDF or Zarr.

Parameters:

  • output_file – Destination artifact path. Saving writes and may overwrite this artifact.

  • output_format – Storage format. When omitted, infer it from a .nc or .zarr suffix. An explicit format adds or replaces the corresponding suffix.

Raises:

ValueError – If metadata is invalid or the output format cannot be inferred.

site_metadata: Dataset#
property sites: tuple[str, ...]#

Return retained site labels in indicator-decoding order.

to_datatree() → DataTree#

Convert prepared RHIME inputs to the versioned DataTree schema.

The basis object is embedded in the artifact. basis_artifact_path is retained only as provenance and is never read while serializing or reconstructing the prepared inputs.

Returns:

DataTree containing inv_inputs, basis_functions, and site_metadata child nodes.

Raises:

ValueError – If site metadata or inversion inputs no longer satisfy the prepared-input invariants.

validated() → Self#

Return a freshly canonicalized copy of these prepared inputs.

This re-establishes the semantic invariants after possible in-place mutation of the contained xarray objects.

Returns:

Prepared inputs normalized from the current xarray values.

openghg_inversions.inversion_data.preparation.prepare_rhime_inputs(*, species: str, sites: list[str], domain: str, averaging_period: Sequence[str | None] | str | None, start_date: str, end_date: str, output_name: str, flux_sources: list[str], split_by_sectors: bool = False, bc_store: str = 'user', obs_store: str = 'user', footprint_store: str = 'user', emissions_store: str = 'user', met_model: Sequence[str | None] | str | None = None, fp_model: str | None = None, fp_height: Sequence[str | None] | str | None = None, fp_species: str | None = None, inlet: Sequence[str | slice | None] | str | None = None, instrument: Sequence[str | None] | str | None = None, max_level: Sequence[int | None] | int | None = None, calibration_scale: str | None = None, obs_data_level: Sequence[str | None] | str | None = None, platform: Sequence[str | None] | str | None = None, use_tracer: bool = False, use_bc: bool = True, fp_basis_case: str | None = None, basis_directory: str | None = None, bc_basis_case: str = 'NESW', bc_basis_directory: str | Path | None = None, country_directory: str | None = None, bc_input: str | None = None, basis_algorithm: str = 'weighted', nbasis: int = 100, filters: Any = None, fix_basis_outer_regions: bool = False, averaging_error: bool = True, bc_freq: str | None = None, reload_merged_data: bool = False, save_merged_data: bool = False, merged_data_dir: str | None = None, merged_data_name: str | None = None, basis_output_path: str | None = None, min_error: Literal['percentile', 'residual'] | dict[str, float] | None | int | float = 0.0, min_error_options: Mapping[str, Any] | None = None, flux_non_finite_check: Literal['lazy', 'count'] = 'lazy') → RhimePreparedInputs#

Prepare modern RHIME inputs without exposing legacy fixedbasis containers.

Observation filters are applied once to merged data before basis loading or generation. The same filtered site datasets and aligned metadata are then used for sensitivity construction.

Parameters:

  • species – Primary gas or tracer name used for object-store lookup and output naming.

  • sites – Requested observation site names.

  • domain – Model domain name.

  • averaging_period – Observation averaging period, either scalar or site-aligned.

  • start_date – Inclusive inversion start date.

  • end_date – Exclusive inversion end date.

  • output_name – Base output name used for data and basis artifacts.

  • flux_sources – OpenGHG flux source values requested for the run.

  • split_by_sectors – Whether to keep sector-resolved sensitivity inputs with a source provenance coordinate. Semantic sector names are applied later by the model specification.

  • inlet – Inlet selector, either scalar or aligned to sites. Entries may be strings, legacy slice selectors, or None.

  • fp_height – Footprint inlet height, either scalar or aligned to sites.

  • instrument – Observation instrument, either scalar or aligned to sites.

  • platform – Observation platform, either scalar or aligned to sites.

  • obs_data_level – Observation data level, either scalar or aligned to sites.

  • met_model – Footprint meteorological model, either scalar or aligned to sites.

  • max_level – Maximum column level, either scalar or aligned to sites. Entries must be integers or None.

  • min_error – Numeric minimum error or "residual"/"percentile" calculation method.

  • min_error_options – Calculated minimum-error options. The only supported key is boolean by_site.

  • use_tracer – Unsupported placeholder for tracer inversions, where an additional species constrains the primary species through linked forward models.

  • flux_non_finite_check – Non-finite flux handling mode. "lazy" applies zero-fill lazily and records attrs; "count" computes count metadata once and warns if non-finite values are present.

Returns:

Modern RHIME prepared inputs containing canonical inv_inputs and a retained BasisFunctions object.

Raises:

ValueError – If site options are empty, duplicated, misaligned, or have invalid types, or if minimum-error options are invalid.