openghg_inversions.basis#
Functions for creating basis functions and applying them to sensitivity matrices.
- openghg_inversions.basis.basis_functions_wrapper(fp_all: dict, species: str, domain: str, start_date: str, emissions_name: list[str] | None, nbasis: int, use_bc: bool, basis_algorithm: str | None = None, fix_outer_regions: bool = False, fp_basis_case: str | None = None, bc_basis_case: str | None = None, basis_directory: str | None = None, bc_basis_directory: str | None = None, country_directory: str | None = None, outputname: str | None = None, output_path: str | None = None, return_basis_objects: bool = False, basis_output_format: Literal['legacy', 'datatree'] = 'legacy', region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None)#
Create basis sensitivities for a legacy fixed-basis inversion.
- Parameters:
fp_all – Legacy merged-data dictionary containing flux and footprint data.
species – Atmospheric trace gas species used when saving generated basis artifacts.
domain – Inversion domain.
start_date – Start date of the inversion period.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Desired number of generated basis regions.
use_bc – If true, include boundary-condition sensitivities.
basis_algorithm – Algorithm used when generating a basis field on the fly.
"quadtree"does not impose land/sea separation,"weighted"uses the legacy weighted land/sea split, and"region_constrained"requires caller-suppliedregion_classesto prevent labels crossing those classes.fix_outer_regions – If true, use fixed InTEM outer regions and generate basis labels only for the inner region.
fp_basis_case – Optional saved emissions basis case. When supplied, a saved artifact is loaded instead of generating a basis.
bc_basis_case – Boundary-condition basis case to load when
use_bcis true.basis_directory – Optional root directory for saved emissions basis artifacts.
bc_basis_directory – Optional root directory for saved boundary-condition basis artifacts.
country_directory – Optional directory containing auxiliary land/sea and InTEM outer-region files used by generated basis algorithms.
outputname – Optional output-name component used when saving generated basis artifacts.
output_path – Optional directory where generated basis artifacts should be saved.
return_basis_objects – If true, return the legacy
fp_datadictionary plus retained basis objects.basis_output_format – Format for saved generated basis artifacts.
"legacy"writes the historical flat netCDF file, while"datatree"writes the retainedBasisFunctionsartifact.region_classes – Two-dimensional class field used only with
basis_algorithm="region_constrained". Loading this field from a file is the caller’s responsibility.region_allocation – Automatic class-allocation mode for
region_constrained. One of"weight"or"area".min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using
region_constrained.split_acceptance – Optional split-acceptance criterion for
region_constrained. Defaults to"none"to preserve existing behavior.contrast_contribution – Design contribution array for contrast scoring; do not use observed mole-fraction values or residuals.
contrast_cell_weight – Optional prior flux or split-mass field for contrast scoring.
min_contrast_delta_eig – Optional minimum contrast
delta_eig.min_contrast_lambda – Optional minimum contrast
lambda.contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted,
tau=1is uncalibrated.contrast_sigma_design – Optional scalar design standard deviation.
contrast_s_diag – Optional diagonal design covariance entries.
- Returns:
By default, returns a dictionary similar to
fp_allbut with basis function and sensitivity data added. Ifreturn_basis_objects=True, returns(fp_data, basis_objects)wherebasis_objects["emissions"]is the retained emissionsBasisFunctionsobject.- Raises:
ValueError – If boundary conditions are requested without
bc_basis_case.
- openghg_inversions.basis.basis_weights_from_fp_all(fp_all: dict, emissions_name: list[str] | None = None, *, abs_flux: bool = False, mask: DataArray | None = None) DataArray#
Build the standard 2D basis weight field from a legacy
fp_allmapping.The generated basis algorithms historically computed weights internally from mean footprints multiplied by mean flux. This helper exposes that adapter step so algorithms and experiments can use already computed weight fields directly.
- Parameters:
fp_all – Legacy merged-data dictionary produced by the data preparation path.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.abs_flux – If true, use absolute flux values before averaging.
mask – Optional Boolean spatial mask. When supplied, weights outside the mask are dropped from the returned field.
- Returns:
Two-dimensional weight field with spatial coordinates preserved.
- openghg_inversions.basis.bucket_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, nbasis: int = 100, country_directory: str | None = None) DataArray#
Create a legacy weighted bucket basis field from precomputed 2D weights.
This is a weight-first version of
bucket_basis_function(). It still delegates to the existing land/sea-aware weighted algorithm for compatibility.- Parameters:
weights – Two-dimensional basis weight field.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
nbasis – Desired number of basis regions.
country_directory – Optional directory containing land/sea files.
- Returns:
Basis field with
lat/londimensions, a singletontimedimension, and integer region labels.
- openghg_inversions.basis.bucket_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, mask: DataArray | None = None) DataArray#
Create a basis field with the legacy weighted bucket algorithm.
This algorithm recursively splits weighted rectangles so each scaling region contains approximately the same total weight. The implementation also uses land/sea masks from
country_directorythrough the lower-level weighted algorithm.- Parameters:
fp_all – Legacy merged-data dictionary produced by the data preparation path.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Desired number of basis regions.
country_directory – Optional directory containing land/sea files. When omitted, default package files are used.
abs_flux – If true, use absolute flux values when constructing weights.
mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.
- Returns:
Basis field with
lat/londimensions, a singletontimedimension, and integer region labels.
- openghg_inversions.basis.bucketbasisfunction(*args, **kwargs) DataArray#
Deprecated alias for
bucket_basis_function().
- openghg_inversions.basis.fixed_outer_regions_basis(fp_all: dict, start_date: str, basis_algorithm: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, *, region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#
Use fixed InTEM outer regions and fit inner regions with an algorithm.
The InTEM outer-region file defines known outer labels. The largest region value is treated as the inner inversion region; this inner mask is passed to
basis_algorithmand then inserted back into the fixed outer map.- Parameters:
fp_all – Legacy merged-data dictionary produced by the data preparation path.
start_date – Start date of the inversion period.
basis_algorithm – Algorithm used to fit the inner region. Supported values are
"quadtree","weighted", and"region_constrained".domain – Domain across which to calculate basis functions.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Desired number of inner-region basis labels.
country_directory – Optional directory containing land/sea files and the InTEM outer-region file. When omitted, default package files are used.
abs_flux – If true, use absolute flux values when constructing weights.
region_classes – Region or country class field used only with
basis_algorithm="region_constrained". File loading should happen before calling this helper.region_allocation – Allocation mode for
region_constrained. One of"weight"or"area".min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using
region_constrained.split_acceptance – Optional split-acceptance criterion for
region_constrainedinner-region splitting.contrast_contribution – Design contribution array used only when
split_acceptance="contrast_score".contrast_cell_weight – Optional prior flux or split-mass field used for contrast scoring.
min_contrast_delta_eig – Optional minimum contrast
delta_eig.min_contrast_lambda – Optional minimum contrast
lambda.contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted,
tau=1is uncalibrated.contrast_sigma_design – Optional scalar design standard deviation.
contrast_s_diag – Optional diagonal design covariance entries.
- Returns:
Basis field with fixed outer labels and generated inner labels.
- openghg_inversions.basis.load_basis_functions(*, fp_all: dict, domain: str, basis_case: str, basis_directory: str | Path | None = None) FluxWeightedBasis#
Load a saved basis artifact as retained
BasisFunctions.DataTree artifacts are preferred when the matching file carries the
openghg_inversions.flux_weighted_basisschema and are loaded throughBasisFunctions.load. Otherwise the existing legacy flat artifact loader is used and a retained basis object is built from runtime flux infp_all.- Parameters:
fp_all – Legacy merged-data dictionary used to build runtime flux when adapting legacy flat artifacts or replacing serialized DataTree flux.
domain – Inversion domain used in the artifact path convention.
basis_case – Basis case prefix used in the artifact path convention.
basis_directory – Optional root directory containing per-domain basis artifact subdirectories.
- Returns:
Loaded retained basis object with
basis_artifact_sourcemetadata.- Raises:
FileNotFoundError – If no matching artifact files are found.
ValueError – If more than one matching DataTree artifact is found.
- openghg_inversions.basis.make_basis_functions(*, fp_all: dict, species: str, domain: str, start_date: str, emissions_name: list[str] | None, nbasis: int, basis_algorithm: str | None = None, fix_outer_regions: bool = False, fp_basis_case: str | None = None, basis_directory: str | None = None, country_directory: str | None = None, outputname: str | None = None, output_path: str | None = None, basis_output_format: Literal['legacy', 'datatree'] = 'legacy', region_classes: DataArray | None = None, region_allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) FluxWeightedBasis#
Create or load retained emissions basis functions.
This helper owns basis artifact generation/loading without applying the legacy fixedbasis
fp_dataside channels.- Parameters:
fp_all – Legacy merged-data dictionary containing flux and footprint data.
species – Atmospheric trace gas species used when saving generated basis artifacts.
domain – Inversion domain used for generated basis metadata and basis artifact lookup.
start_date – Start date of the inversion period.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Desired number of generated basis regions.
basis_algorithm – Algorithm used when generating a basis field on the fly. Supported values are
"quadtree","weighted", and"region_constrained".fix_outer_regions – If true, use fixed InTEM outer regions and generate basis labels only for the inner region.
fp_basis_case – Optional saved emissions basis case. When supplied, a saved artifact is loaded instead of generating a basis.
basis_directory – Optional root directory for saved emissions basis artifacts.
country_directory – Optional directory containing auxiliary land/sea and InTEM outer-region files used by generated basis algorithms.
outputname – Optional output-name component used when saving generated basis artifacts.
output_path – Optional directory where generated basis artifacts should be saved.
basis_output_format – Format for saved generated basis artifacts.
"legacy"writes the historical flat netCDF file, while"datatree"writes the retainedBasisFunctionsartifact.region_classes – Two-dimensional class field used only with
basis_algorithm="region_constrained". Loading this field from a file is the caller’s responsibility.region_allocation – Automatic class-allocation mode for
region_constrained. One of"weight"or"area".min_regions_per_class – Minimum automatic allocation for each non-empty mapped class when using
region_constrained.split_acceptance – Optional split-acceptance criterion for
region_constrained. Defaults to"none"to preserve existing behavior."contrast_score"requirescontrast_contribution.contrast_contribution – Design contribution array for contrast scoring; do not use observed mole-fraction values or residuals.
contrast_cell_weight – Optional prior flux or split-mass field for contrast scoring.
min_contrast_delta_eig – Optional minimum contrast
delta_eig.min_contrast_lambda – Optional minimum contrast
lambda.contrast_tau – Prior standard deviation of the split contrast coefficient. If omitted,
tau=1is uncalibrated.contrast_sigma_design – Optional scalar design standard deviation.
contrast_s_diag – Optional diagonal design covariance entries.
- Returns:
Retained emissions basis object ready for sensitivity projection.
- Raises:
ValueError – If neither a saved basis case nor an algorithm is supplied, or if an unsupported output format or basis algorithm is requested.
TypeError – If a generated algorithm returns an unsupported basis object.
- openghg_inversions.basis.quadtree_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, nbasis: int = 100, seed: int | None = None) DataArray#
Create a quadtree basis field from precomputed 2D weights.
- Parameters:
weights – Two-dimensional basis weight field.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
nbasis – Desired number of basis regions.
seed – Optional seed passed to
scipy.optimize.dual_annealing.
- Returns:
Basis field with
lat/londimensions, a singletontimedimension, and integer region labels.
- openghg_inversions.basis.quadtree_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int = 100, country_directory: str | None = None, abs_flux: bool = False, seed: int | None = None, mask: DataArray | None = None) DataArray#
Create a basis field with the quadtree algorithm.
The domain is split with smaller grid cells for regions which contribute more to the a priori above-baseline mole fraction. This is based on the average footprint over the inversion period and the a priori emissions field.
The number of basis functions is optimised using dual annealing. Probably not the best or fastest method as there should only be one minimum, but it does not require the Jacobian or Hessian for optimisation.
- Parameters:
fp_all – Legacy merged-data dictionary produced by the data preparation path.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Desired number of basis regions.
country_directory – Accepted for a consistent basis-algorithm interface; the quadtree algorithm does not use it.
abs_flux – If true, use absolute flux values when constructing weights.
seed – Optional seed passed to
scipy.optimize.dual_annealing.mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.
- Returns:
Basis field with
lat/londimensions, a singletontimedimension, and integer region labels.
- openghg_inversions.basis.quadtreebasisfunction(*args, **kwargs) DataArray#
Deprecated alias for
quadtree_basis_function().
- openghg_inversions.basis.region_constrained_basis_from_weights(weights: DataArray, start_date: str, domain: str, *, region_classes: DataArray, nbasis: int | Mapping[Hashable, int] = 100, allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#
Create constrained basis labels from weights and region classes.
This is the weight-first adapter for the current
region_constrainedbasis algorithm. Labels are generated independently within each non-null region class.- Parameters:
weights – Two-dimensional basis weight field.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
region_classes – Two-dimensional class field on the same spatial grid as
weights.nbasis – Total number of basis regions, or class-local allocation accepted by
region_constrained_basis.allocation – Automatic allocation mode used when
nbasisis an integer."weight"allocates by class total weight;"area"allocates by mapped cell count.min_regions_per_class – Minimum automatic allocation for each non-empty mapped class.
split_acceptance – Optional split-acceptance criterion. The default
"none"preserves existing behavior."contrast_score"uses a mass-preserving observation-space contrast gate.contrast_contribution – Design contribution array for contrast scoring.
contrast_cell_weight – Optional prior flux or split-mass field used as
muin contrast scoring. When omitted, the unnormalised input weights are used as a split-mass proxy.min_contrast_delta_eig – Optional minimum
delta_eigthreshold.min_contrast_lambda – Optional minimum
lambdathreshold.contrast_tau – Prior standard deviation of the split contrast coefficient
delta = alpha_A - alpha_B.contrast_sigma_design – Optional scalar design standard deviation, equivalent to
S = contrast_sigma_design**2 I.contrast_s_diag – Optional diagonal design covariance entries in the same row space as
contrast_contribution.
- Returns:
Basis field with globally unique integer labels that do not cross
region_classesvalues.
- openghg_inversions.basis.region_constrained_basis_function(fp_all: dict, start_date: str, domain: str, emissions_name: list[str] | None = None, nbasis: int | Mapping[Hashable, int] = 100, country_directory: str | None = None, abs_flux: bool = False, mask: DataArray | None = None, region_classes: DataArray | None = None, allocation: Literal['weight', 'area'] = 'weight', min_regions_per_class: int = 1, split_acceptance: Literal['none', 'contrast_score'] = 'none', contrast_contribution: DataArray | None = None, contrast_cell_weight: DataArray | None = None, min_contrast_delta_eig: float | None = None, min_contrast_lambda: float | None = None, contrast_tau: float | None = None, contrast_sigma_design: float | None = None, contrast_s_diag: DataArray | None = None) DataArray#
Create weighted basis regions constrained by caller-supplied classes.
This adapter keeps file loading outside the constrained algorithm: callers provide
region_classesdirectly, for example from a country file, land/sea file, or a user-defined region-class field. It links the pureregion_constrained_basishelper to the currentfp_all-based wrapper interface by constructing the usual footprint-times-flux weight field first.- Parameters:
fp_all – Legacy merged-data dictionary produced by the data preparation path.
start_date – Start date of the inversion period.
domain – Domain across which to calculate basis functions.
emissions_name – Optional list of OpenGHG flux source names used to select emissions from
fp_all.nbasis – Total number of basis regions, or class-local allocation accepted by
region_constrained_basis.country_directory – Accepted for a consistent basis-algorithm interface; file loading for
region_classesmust happen before calling this adapter.abs_flux – If true, use absolute flux values when constructing weights.
mask – Optional Boolean spatial mask for fitting basis functions over a sub-region.
region_classes – Two-dimensional class field on the same spatial grid as the generated weights. Positive basis labels are generated independently within each non-null class value.
allocation – Automatic allocation mode used when
nbasisis an integer."weight"allocates regions by total class weight;"area"allocates by mapped cell count.min_regions_per_class – Minimum automatic allocation for each non-empty mapped class.
split_acceptance – Optional split-acceptance criterion. The default
"none"preserves existing behavior."contrast_score"uses a mass-preserving observation-space contrast gate.contrast_contribution – Design contribution array for contrast scoring, with at least one design-observation dimension plus the two spatial dimensions. Observed mole-fraction values must not be used here.
contrast_cell_weight – Optional prior flux or split-mass field used as
muin contrast scoring. When omitted, the unnormalised generated basis weight field is used as a split-mass proxy.min_contrast_delta_eig – Optional minimum
delta_eigthreshold. If both contrast thresholds are omitted, contrast diagnostics are computed but proposed splits are not rejected.min_contrast_lambda – Optional minimum
lambdathreshold.contrast_tau – Prior standard deviation of the split contrast coefficient
delta = alpha_A - alpha_B. If omitted,tau=1is used and scores are uncalibrated.contrast_sigma_design – Optional scalar design standard deviation, equivalent to
S = contrast_sigma_design**2 I.contrast_s_diag – Optional diagonal design covariance entries in the same row space as
contrast_contribution.
- Returns:
Basis field with
lat/londimensions, a singletontimedimension, and globally unique integer labels that do not crossregion_classesvalues.- Raises:
ValueError – If
region_classesis not supplied.
- openghg_inversions.basis.algorithms
AllSplitAcceptancePoliciesAxisAlignedWeightedSplitStrategyAxisParallelSplitStepContrastScoreSplitAcceptanceContrastScoreSplitAcceptance.accepts()ContrastScoreSplitAcceptance.cell_weightContrastScoreSplitAcceptance.contrast_s_diagContrastScoreSplitAcceptance.contrast_sigma_designContrastScoreSplitAcceptance.contrast_tauContrastScoreSplitAcceptance.contributionContrastScoreSplitAcceptance.min_contrast_delta_eigContrastScoreSplitAcceptance.min_contrast_lambdaContrastScoreSplitAcceptance.score_split()ContrastScoreSplitAcceptance.spatial_dims
GreedyAxisParallelSplitStrategyInertialSplitStepLatLonGridGeometryMaxChildPCAEccentricityMinChildTargetWeightShareMinChildWeightSharePartitionStepSplitAcceptancePolicySplitContrastScoreSplitGeometrySplitStrategyTargetSplitAcceptancePolicyallocate_nbasis_by_class()contrast_tau_from_multiplier_cv()intersect_region_class_layers()quadtree_algorithm()region_constrained_basis()split_contrast_score()weighted_algorithm()
- openghg_inversions.basis.basis_functions
BasisFunctionsFluxWeightedBasisFluxWeightedBasis.basis_artifact_pathFluxWeightedBasis.basis_artifact_sourceFluxWeightedBasis.flat_basis()FluxWeightedBasis.fluxFluxWeightedBasis.for_source()FluxWeightedBasis.from_datatree()FluxWeightedBasis.from_flat_basis()FluxWeightedBasis.from_multi_source_flat_basis()FluxWeightedBasis.interpolate()FluxWeightedBasis.load()FluxWeightedBasis.metadataFluxWeightedBasis.operatorFluxWeightedBasis.plot()FluxWeightedBasis.save()FluxWeightedBasis.select_sources()FluxWeightedBasis.sensitivity()FluxWeightedBasis.to_datatree()FluxWeightedBasis.with_flux()FluxWeightedBasis.with_metadata()
basis_functions_from_fp_all_flat_basis()flux_from_fp_all()
- openghg_inversions.basis.layout
- openghg_inversions.basis.operators
- Design goals
- How to use
BasisMetaBasisOperatorBucketBasisOperatorMultiSourceBucketBasisOperatorMultiSourceBucketBasisOperator.basis_matrixMultiSourceBucketBasisOperator.from_datatree()MultiSourceBucketBasisOperator.interpolate()MultiSourceBucketBasisOperator.kindMultiSourceBucketBasisOperator.metaMultiSourceBucketBasisOperator.operator_for_source()MultiSourceBucketBasisOperator.sensitivity()MultiSourceBucketBasisOperator.to_datatree()
drop_singleton_time()get_basis_operator_class()register_basis_operator()