openghg_inversions.basis#

Functions for creating basis functions and applying them to sensitivity matrices.

openghg_inversions.basis.basis_functions_wrapper(fp_all: dict, species: str, domain: str, start_date: str, emissions_name: list[str] | None, nbasis: int, use_bc: bool, basis_algorithm: str | None = None, fix_outer_regions: bool = False, fp_basis_case: str | None = None, bc_basis_case: str | None = None, basis_directory: str | None = None, bc_basis_directory: str | None = None, country_directory: str | None = None, outputname: str | None = None, output_path: str | None = None, return_basis_objects: bool = False, basis_output_format: Literal['legacy', 'datatree'] = 'legacy', region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None)#

Create basis sensitivities for a legacy fixed-basis inversion.

Parameters:
  • fp_all – Legacy merged-data dictionary containing flux and footprint data.

  • species – Atmospheric trace gas species used when saving generated basis artifacts.

  • domain – Inversion domain.

  • start_date – Start date of the inversion period.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Desired number of generated basis regions.

  • use_bc – If true, include boundary-condition sensitivities.

  • basis_algorithm – Algorithm used when generating a basis field on the fly. "quadtree" does not impose land/sea separation, "weighted" uses the legacy weighted land/sea split, and "region_constrained" requires caller-supplied region_classes to prevent labels crossing those classes.

  • fix_outer_regions – If true, use fixed InTEM outer regions and generate basis labels only for the inner region.

  • fp_basis_case – Optional saved emissions basis case. When supplied, a saved artifact is loaded instead of generating a basis.

  • bc_basis_case – Boundary-condition basis case to load when use_bc is true.

  • basis_directory – Optional root directory for saved emissions basis artifacts.

  • bc_basis_directory – Optional root directory for saved boundary-condition basis artifacts.

  • country_directory – Optional directory containing auxiliary land/sea and InTEM outer-region files used by generated basis algorithms.

  • outputname – Optional output-name component used when saving generated basis artifacts.

  • output_path – Optional directory where generated basis artifacts should be saved.

  • return_basis_objects – If true, return the legacy fp_data dictionary plus retained basis objects.

  • basis_output_format – Format for saved generated basis artifacts. "legacy" writes the historical flat netCDF file, while "datatree" writes the retained BasisFunctions artifact.

  • region_classes – Two-dimensional class field used only with basis_algorithm="region_constrained". Loading this field from a file is the caller’s responsibility.

  • region_allocation – Automatic class-allocation mode for region_constrained. One of "weight" or "area".

  • min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using region_constrained.

  • split_acceptance – Optional split-acceptance criterion for region_constrained. Defaults to "none" to preserve existing behavior.

  • contrast_contribution – Design contribution array for contrast scoring; do not use observed mole-fraction values or residuals.

  • contrast_cell_weight – Optional prior flux or split-mass field for contrast scoring.

  • min_contrast_delta_eig – Optional minimum contrast delta_eig.

  • min_contrast_lambda – Optional minimum contrast lambda.

  • contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted, tau=1 is uncalibrated.

  • contrast_sigma_design – Optional scalar design standard deviation.

  • contrast_s_diag – Optional diagonal design covariance entries.

Returns:

By default, returns a dictionary similar to fp_all but with basis function and sensitivity data added. If return_basis_objects=True, returns (fp_data, basis_objects) where basis_objects["emissions"] is the retained emissions BasisFunctions object.

Raises:

ValueError – If boundary conditions are requested without bc_basis_case.

openghg_inversions.basis.basis_weights_from_fp_all(fp_all: dict, emissions_name: list[str] | None = None, *, abs_flux: bool = False, mask: DataArray | None = None) DataArray#

Build the standard 2D basis weight field from a legacy fp_all mapping.

The generated basis algorithms historically computed weights internally from mean footprints multiplied by mean flux. This helper exposes that adapter step so algorithms and experiments can use already computed weight fields directly.

Parameters:
  • fp_all – Legacy merged-data dictionary produced by the data preparation path.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • abs_flux – If true, use absolute flux values before averaging.

  • mask – Optional Boolean spatial mask. When supplied, weights outside the mask are dropped from the returned field.

Returns:

Two-dimensional weight field with spatial coordinates preserved.

openghg_inversions.basis.bucket_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, nbasis: int = 100, country_directory: str | None = None) DataArray#

Create a legacy weighted bucket basis field from precomputed 2D weights.

This is a weight-first version of bucket_basis_function(). It still delegates to the existing land/sea-aware weighted algorithm for compatibility.

Parameters:
  • weights – Two-dimensional basis weight field.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • nbasis – Desired number of basis regions.

  • country_directory – Optional directory containing land/sea files.

Returns:

Basis field with lat/lon dimensions, a singleton time dimension, and integer region labels.

openghg_inversions.basis.bucket_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, mask: DataArray | None = None) DataArray#

Create a basis field with the legacy weighted bucket algorithm.

This algorithm recursively splits weighted rectangles so each scaling region contains approximately the same total weight. The implementation also uses land/sea masks from country_directory through the lower-level weighted algorithm.

Parameters:
  • fp_all – Legacy merged-data dictionary produced by the data preparation path.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Desired number of basis regions.

  • country_directory – Optional directory containing land/sea files. When omitted, default package files are used.

  • abs_flux – If true, use absolute flux values when constructing weights.

  • mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.

Returns:

Basis field with lat/lon dimensions, a singleton time dimension, and integer region labels.

openghg_inversions.basis.bucketbasisfunction(*args, **kwargs) DataArray#

Deprecated alias for bucket_basis_function().

openghg_inversions.basis.fixed_outer_regions_basis(fp_all: dict, start_date: str, basis_algorithm: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, *, region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#

Use fixed InTEM outer regions and fit inner regions with an algorithm.

The InTEM outer-region file defines known outer labels. The largest region value is treated as the inner inversion region; this inner mask is passed to basis_algorithm and then inserted back into the fixed outer map.

Parameters:
  • fp_all – Legacy merged-data dictionary produced by the data preparation path.

  • start_date – Start date of the inversion period.

  • basis_algorithm – Algorithm used to fit the inner region. Supported values are "quadtree", "weighted", and "region_constrained".

  • domain – Domain across which to calculate basis functions.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Desired number of inner-region basis labels.

  • country_directory – Optional directory containing land/sea files and the InTEM outer-region file. When omitted, default package files are used.

  • abs_flux – If true, use absolute flux values when constructing weights.

  • region_classes – Region or country class field used only with basis_algorithm="region_constrained". File loading should happen before calling this helper.

  • region_allocation – Allocation mode for region_constrained. One of "weight" or "area".

  • min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using region_constrained.

  • split_acceptance – Optional split-acceptance criterion for region_constrained inner-region splitting.

  • contrast_contribution – Design contribution array used only when split_acceptance="contrast_score".

  • contrast_cell_weight – Optional prior flux or split-mass field used for contrast scoring.

  • min_contrast_delta_eig – Optional minimum contrast delta_eig.

  • min_contrast_lambda – Optional minimum contrast lambda.

  • contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted, tau=1 is uncalibrated.

  • contrast_sigma_design – Optional scalar design standard deviation.

  • contrast_s_diag – Optional diagonal design covariance entries.

Returns:

Basis field with fixed outer labels and generated inner labels.

openghg_inversions.basis.load_basis_functions(*, fp_all: dict, domain: str, basis_case: str, basis_directory: str | Path | None = None) FluxWeightedBasis#

Load a saved basis artifact as retained BasisFunctions.

DataTree artifacts are preferred when the matching file carries the openghg_inversions.flux_weighted_basis schema and are loaded through BasisFunctions.load. Otherwise the existing legacy flat artifact loader is used and a retained basis object is built from runtime flux in fp_all.

Parameters:
  • fp_all – Legacy merged-data dictionary used to build runtime flux when adapting legacy flat artifacts or replacing serialized DataTree flux.

  • domain – Inversion domain used in the artifact path convention.

  • basis_case – Basis case prefix used in the artifact path convention.

  • basis_directory – Optional root directory containing per-domain basis artifact subdirectories.

Returns:

Loaded retained basis object with basis_artifact_source metadata.

Raises:
openghg_inversions.basis.make_basis_functions(*, fp_all: dict, species: str, domain: str, start_date: str, emissions_name: list[str] | None, nbasis: int, basis_algorithm: str | None = None, fix_outer_regions: bool = False, fp_basis_case: str | None = None, basis_directory: str | None = None, country_directory: str | None = None, outputname: str | None = None, output_path: str | None = None, basis_output_format: Literal['legacy', 'datatree'] = 'legacy', region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) FluxWeightedBasis#

Create or load retained emissions basis functions.

This helper owns basis artifact generation/loading without applying the legacy fixedbasis fp_data side channels.

Parameters:
  • fp_all – Legacy merged-data dictionary containing flux and footprint data.

  • species – Atmospheric trace gas species used when saving generated basis artifacts.

  • domain – Inversion domain used for generated basis metadata and basis artifact lookup.

  • start_date – Start date of the inversion period.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Desired number of generated basis regions.

  • basis_algorithm – Algorithm used when generating a basis field on the fly. Supported values are "quadtree", "weighted", and "region_constrained".

  • fix_outer_regions – If true, use fixed InTEM outer regions and generate basis labels only for the inner region.

  • fp_basis_case – Optional saved emissions basis case. When supplied, a saved artifact is loaded instead of generating a basis.

  • basis_directory – Optional root directory for saved emissions basis artifacts.

  • country_directory – Optional directory containing auxiliary land/sea and InTEM outer-region files used by generated basis algorithms.

  • outputname – Optional output-name component used when saving generated basis artifacts.

  • output_path – Optional directory where generated basis artifacts should be saved.

  • basis_output_format – Format for saved generated basis artifacts. "legacy" writes the historical flat netCDF file, while "datatree" writes the retained BasisFunctions artifact.

  • region_classes – Two-dimensional class field used only with basis_algorithm="region_constrained". Loading this field from a file is the caller’s responsibility.

  • region_allocation – Automatic class-allocation mode for region_constrained. One of "weight" or "area".

  • min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using region_constrained.

  • split_acceptance – Optional split-acceptance criterion for region_constrained. Defaults to "none" to preserve existing behavior. "contrast_score" requires contrast_contribution.

  • contrast_contribution – Design contribution array for contrast scoring; do not use observed mole-fraction values or residuals.

  • contrast_cell_weight – Optional prior flux or split-mass field for contrast scoring.

  • min_contrast_delta_eig – Optional minimum contrast delta_eig.

  • min_contrast_lambda – Optional minimum contrast lambda.

  • contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted, tau=1 is uncalibrated.

  • contrast_sigma_design – Optional scalar design standard deviation.

  • contrast_s_diag – Optional diagonal design covariance entries.

Returns:

Retained emissions basis object ready for sensitivity projection.

Raises:
  • ValueError – If neither a saved basis case nor an algorithm is supplied, or if an unsupported output format or basis algorithm is requested.

  • TypeError – If a generated algorithm returns an unsupported basis object.

openghg_inversions.basis.quadtree_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, nbasis: int = 100, seed: int | None = None) DataArray#

Create a quadtree basis field from precomputed 2D weights.

Parameters:
  • weights – Two-dimensional basis weight field.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • nbasis – Desired number of basis regions.

  • seed – Optional seed passed to scipy.optimize.dual_annealing.

Returns:

Basis field with lat/lon dimensions, a singleton time dimension, and integer region labels.

openghg_inversions.basis.quadtree_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, seed: int | None = None, mask: DataArray | None = None) DataArray#

Create a basis field with the quadtree algorithm.

The domain is split with smaller grid cells for regions which contribute more to the a priori above-baseline mole fraction. This is based on the average footprint over the inversion period and the a priori emissions field.

The number of basis functions is optimised using dual annealing. Probably not the best or fastest method as there should only be one minimum, but it does not require the Jacobian or Hessian for optimisation.

Parameters:
  • fp_all – Legacy merged-data dictionary produced by the data preparation path.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Desired number of basis regions.

  • country_directory – Accepted for a consistent basis-algorithm interface; the quadtree algorithm does not use it.

  • abs_flux – If true, use absolute flux values when constructing weights.

  • seed – Optional seed passed to scipy.optimize.dual_annealing.

  • mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.

Returns:

Basis field with lat/lon dimensions, a singleton time dimension, and integer region labels.

openghg_inversions.basis.quadtreebasisfunction(*args, **kwargs) DataArray#

Deprecated alias for quadtree_basis_function().

openghg_inversions.basis.region_constrained_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, region_classes: DataArray, nbasis: int | Mapping[Hashable, int] = 100, allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#

Create constrained basis labels from weights and region classes.

This is the weight-first adapter for the current region_constrained basis algorithm. Labels are generated independently within each non-null region class.

Parameters:
  • weights – Two-dimensional basis weight field.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • region_classes – Two-dimensional class field on the same spatial grid as weights.

  • nbasis – Total number of basis regions, or class-local allocation accepted by region_constrained_basis.

  • allocation – Automatic allocation mode used when nbasis is an integer. "weight" allocates by class total weight; "area" allocates by mapped cell count.

  • min_regions_per_class – Minimum automatic allocation for each non-empty mapped class.

  • split_acceptance – Optional split-acceptance criterion. The default "none" preserves existing behavior. "contrast_score" uses a mass-preserving observation-space contrast gate.

  • contrast_contribution – Design contribution array for contrast scoring.

  • contrast_cell_weight – Optional prior flux or split-mass field used as mu in contrast scoring. When omitted, the unnormalised input weights are used as a split-mass proxy.

  • min_contrast_delta_eig – Optional minimum delta_eig threshold.

  • min_contrast_lambda – Optional minimum lambda threshold.

  • contrast_tau – Prior standard deviation of the split contrast coefficient delta = alpha_A - alpha_B.

  • contrast_sigma_design – Optional scalar design standard deviation, equivalent to S = contrast_sigma_design**2 I.

  • contrast_s_diag – Optional diagonal design covariance entries in the same row space as contrast_contribution.

Returns:

Basis field with globally unique integer labels that do not cross region_classes values.

openghg_inversions.basis.region_constrained_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int | Mapping[Hashable, int] = 100, country_directory: str | None = None, abs_flux: bool = False, mask: DataArray | None = None, region_classes: DataArray | None = None, allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#

Create weighted basis regions constrained by caller-supplied classes.

This adapter keeps file loading outside the constrained algorithm: callers provide region_classes directly, for example from a country file, land/sea file, or a user-defined region-class field. It links the pure region_constrained_basis helper to the current fp_all-based wrapper interface by constructing the usual footprint-times-flux weight field first.

Parameters:
  • fp_all – Legacy merged-data dictionary produced by the data preparation path.

  • start_date – Start date of the inversion period.

  • domain – Domain across which to calculate basis functions.

  • emissions_name – Optional list of OpenGHG flux source names used to select emissions from fp_all.

  • nbasis – Total number of basis regions, or class-local allocation accepted by region_constrained_basis.

  • country_directory – Accepted for a consistent basis-algorithm interface; file loading for region_classes must happen before calling this adapter.

  • abs_flux – If true, use absolute flux values when constructing weights.

  • mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.

  • region_classes – Two-dimensional class field on the same spatial grid as the generated weights. Positive basis labels are generated independently within each non-null class value.

  • allocation – Automatic allocation mode used when nbasis is an integer. "weight" allocates regions by total class weight; "area" allocates by mapped cell count.

  • min_regions_per_class – Minimum automatic allocation for each non-empty mapped class.

  • split_acceptance – Optional split-acceptance criterion. The default "none" preserves existing behavior. "contrast_score" uses a mass-preserving observation-space contrast gate.

  • contrast_contribution – Design contribution array for contrast scoring, with at least one design-observation dimension plus the two spatial dimensions. Observed mole-fraction values must not be used here.

  • contrast_cell_weight – Optional prior flux or split-mass field used as mu in contrast scoring. When omitted, the unnormalised generated basis weight field is used as a split-mass proxy.

  • min_contrast_delta_eig – Optional minimum delta_eig threshold. If both contrast thresholds are omitted, contrast diagnostics are computed but proposed splits are not rejected.

  • min_contrast_lambda – Optional minimum lambda threshold.

  • contrast_tau – Prior standard deviation of the split contrast coefficient delta = alpha_A - alpha_B. If omitted, tau=1 is used and scores are uncalibrated.

  • contrast_sigma_design – Optional scalar design standard deviation, equivalent to S = contrast_sigma_design**2 I.

  • contrast_s_diag – Optional diagonal design covariance entries in the same row space as contrast_contribution.

Returns:

Basis field with lat/lon dimensions, a singleton time dimension, and globally unique integer labels that do not cross region_classes values.

Raises:

ValueError – If region_classes is not supplied.