openghg_inversions.inversion_data#

openghg_inversions.inversion_data.data_processing_surface_notracer(species: str, sites: list | str, domain: str, averaging_period: list[str | None] | str | None, start_date: str, end_date: str, obs_data_level: list[str | None] | str | None = None, platform: list[str | None] | str | None = None, inlet: list[str | None] | str | None = None, instrument: list[str | None] | str | None = None, max_level: int | None = None, calibration_scale: str | None = None, met_model: list[str | None] | str | None = None, fp_model: str | None = None, fp_height: list[str | None | Literal['auto']] | Literal['auto'] | str | None = None, fp_species: str | None = None, emissions_name: list | None = None, use_bc: bool = True, bc_input: str | None = None, bc_store: str | None = None, obs_store: str | list[str] | None = None, footprint_store: str | list[str] | None = None, emissions_store: str | None = None, averagingerror: bool = True, save_merged_data: bool = False, merged_data_name: str | None = None, merged_data_dir: str | None = None, output_name: str | None = None) → tuple[dict, list, list, list, list, list]#

Retrieve and prepare fixed-surface datasets from specified OpenGHG object stores.

Use for forward simulations and model-data comparisons that do not use tracers.

Parameters:

  • species – Atmospheric trace gas species of interest e.g. “co2”

  • sites – List of strings containing measurement station/site abbreviations e.g. [“MHD”, “TAC”] NOTE: for satellite, pass as “satellitename-obs_region” eg “GOSAT-BRAZIL” and pass corresponding platform as “satellite”

  • domain – Model domain region of interest; e.g. “EUROPE”

  • averaging_period – List of averaging periods to apply to mole fraction data. NB. len(averaging_period)==len(sites) e.g. [“1H”, “1H”]

  • start_date – Date from which to gather data; e.g. “2020-01-01”

  • end_date – Date until which to gather data; e.g. “2020-02-01”

  • obs_data_level – ICOS observations data level. For non-ICOS sites use “None”

  • inlet – Specific inlet height for the site observations (length must match number of sites)

  • instrument – Specific instrument for the site (length must match number of sites)

  • max_level – Maximum atmospheric level to extract. Only needed if using satellite data.

  • calibration_scale – Convert measurements to defined calibration scale

  • met_model – Meteorological model used in the LPDM. List must be same length as number of sites.

  • fp_model – LPDM used for generating footprints.

  • fp_height – Inlet height used in footprints for corresponding sites.

  • fp_species – Species name associated with footprints in the object store

  • emissions_name – List of keywords args associated with emissions files in the object store. Corresponds to source in OpenGHG.

  • use_bc – Option to include boundary conditions in model

  • bc_input – Variable for calling BC data from ‘bc_store’ - equivalent of ‘emissions_name’ for fluxes.

  • bc_store – Name of object store to retrieve boundary conditions data from.

  • obs_store – Name of object store to retrieve observations data from.

  • footprint_store – Name of object store to retrieve footprints data from.

  • emissions_store – Name of object store to retrieve emissions data from.

  • averagingerror – Adds the variability in the averaging period to the measurement error if set to True.

  • save_merged_data – Save forward simulations data and observations.

  • merged_data_name – Filename for saved forward simulations data and observations.

  • merged_data_dir – Directory path for for saved forward simulations data and observations.

  • output_name – Optional name used to create merged data name.

Returns:

containing

  • fp_all: dictionary containing flux data (key “.flux”), bc data (key “.bc”), and observations data (site short name as key)

  • sites: Updated list of sites. All put in upper case and if data was not extracted correctly for any sites, drop these from the rest of the inversion.

  • inlet: List of inlet height for the updated list of sites

  • fp_height: List of footprint height for the updated list of sites

  • instrument: List of instrument for the updated list of sites

  • averaging_period: List of averaging_period for the updated list of sites

Return type:

tuple

openghg_inversions.inversion_data.load_merged_data(merged_data_dir: str | Path, species: str | None = None, start_date: str | None = None, output_name: str | None = None, merged_data_name: str | None = None, output_format: Literal['pickle', 'netcdf', 'zarr', 'zarr.zip'] | None = None) → dict#

Load fp_all dictionary from a file in merged_data_dir.

The name of the pickle file can be specified using merged_data_name, or a standard name will be created given species, start_date, and output_name.

If merged_data_name is not given, then species, start_date, and output_name must be provided.

This function tries to automatically find a compatible format of merged data, if a format is not specified. First, it checks for data in “zarr” format, then in netCDF, and finally in pickle.

Parameters:

  • merged_data_dir – path to directory where merged data will be saved

  • species – species of inversion

  • start_date – start date of inversion period

  • output_name – output name parameter used for inversion run

  • merged_data_name – name to use for saved data.

  • output_format – format of data to load (if not specified, this will be inferred).

Returns:

fp_all dictionary