openghg_inversions.config.config#

Configuration file utilities for INI format files.

This module allows configuration files in the INI format to be read and used.

Example of a section in the configuration file:

[MEASUREMENTS]
# Measurement details

sites = ["GSN"]           ; Sites to read the data from as a list
species = "chcl3"
start_date = "2015-01-01" ; Default start date used if none specified on the command line
end_date = "2015-02-01"   ; Default start date used if none specified on the command line
domain = "EASTASIA"
network = "AGAGE"

Configuration file format:

  • Sections are included in square brackets

  • Parameter name and value pairs are separated by an equals sign

  • Values can be specified with the same syntax as when creating a python object e.g. ‘’ for string, [] for lists (and also for np.array - will be converted if their type has been set as an array)

  • ; and # symbols can be used to create new line and inline comments

Section headings can be of the form [NAME] or [GROUP.NAME]. This allows parameters to be separated into several section headings in the configuration file for clarity but grouped into one overall classification when inputted based on the GROUP name.

param_type dictionary:

To specify inputs and the types they should be cast to (e.g. str, array, boolean etc) a nested dictionary should be created. This can be passed into several functions as the param_type argument.

This should be of the form of one of the following:

  • {‘SECTION_GROUP1’:{‘param1’:str,’param2’:float},’SECTION_GROUP2’:{‘param3’:list,’param4’:np.array}}

  • {‘SECTION1’:{‘param1’:str},’SECTION2’:’{param2’:float},’SECTION3’:{‘param3’:list},’SECTION4’:{‘param4’:np.array}}

  • OrderedDict([‘SECTION_GROUP1’:OrderedDict([(‘param1’:str),(‘param2’:float)]),’SECTION_GROUP2’:OrderedDict([(‘param3’:list),(‘param4’:np.array)]))

  • OrderedDict([‘SECTION1’:{‘param1’:str},’SECTION2’:{‘param2’:float},’SECTION3’:{‘param3’:list},’SECTION4’:{‘param4’:np.array}])

This can either be created directly or a template configuration file can be created and a param_type dictionary created from this using the generate_param_dict() function. These template files should be kept within the acrg_config/templates/ directory and, after creation, should not be altered unless you wish to change the format for all config files of this type.

Note: if param_type is not defined, the code will attempt to cast the inputs to the most sensible type. This should be fine in most cases but could cause issues. This also means the the set of input parameters will not be checked for any missing values.

How to run:

The main functions to use for reading in parameters from a config file are:

  • all_param(config_file,…) ; Extract all parameters from a configuration file.

  • extract_params(config_file,…) ; Extract specific parameters from a file either based on parameter names, sections or groups.

A param_type dictionary can be defined both to fix expected inputs and to explictly specify the parameter types.

@author: rt17603

openghg_inversions.config.config.all_param(config_file, expected_param=[], param_type=None, exclude_not_found=False, allow_new=False)#

Extract all parameters from a config file.

If param_type specified will cast to the specified types, otherwise will attempt to discern the parameter types from the form of the values.

Parameters:

  • config_file – Filename for input configuration file.

  • expected_param – Parameters within the configuration file which must be returned. An error will be raised if these parameters are not present.

  • param_type – Nested dictionary of sections or groups and expected parameter names and types. See module header for expected formats of param_type dictionary.

  • exclude_not_found – Whether to remove parameters which are not found in the input file or include them as None.

  • allow_new – If a param_type is specified, whether to allow unrecognised parameters to be added without printing a warning. Default = False (i.e. Warning will be printed).

Returns:

Parameter names and values.

Return type:

OrderedDict

openghg_inversions.config.config.all_parameters_in_param_type(param_type)#

The all_parameters_in_param_type function extracts all parameters (regardless of section/section_group) for a given param_type nested dictionary.

Parameters:

param_type (dict) – Nested dictionary of expected parameter names and types. Key for each parameter dictionary can be the section heading or the overall group (e.g. for [MCMC.MEASUREMENTS], section group should be ‘MCMC’). See module header for expected formats of param_type dictionary.

Returns:

list of all parameters in param_type dictionary

Return type:

list

openghg_inversions.config.config.convert(string, value_type=None)#

Convert the input string to the specified value_type.

If no value_type is stated, the function attempts to discern the appropriate type.

Parameters:

  • string – Value input from config file.

  • value_type – Object type. Values accepted: “str”, “list”. Optional.

Returns:

Python object of specified type, or None if unable to convert to value_type or find a suitable type when value_type is not specified.

openghg_inversions.config.config.eval_check(string, error=True)#

Evaluate the input string from a configuration file to a python object.

Examples

  • ‘1’ would evalute to an int object 1

  • ‘1.’ or ‘1.0’ would evaluate to float object 1.0

  • “[1,2,3]” would evalute to a list object [1,2,3]

  • “1,2,3” would evaluate to a tuple object (1,2,3)

See eval() documentation for full list.

Parameters:

  • string – Value input from config file.

  • error – Print error message if unable to evaluate.

Returns:

Python object and boolean indicating success.

If unable to convert string to python object, returns (None, False).

Return type:

tuple

openghg_inversions.config.config.extract_params(config_file, expected_param=[], section=None, section_group=None, names=[], ignore_sections=[], ignore_section_groups=[], exclude_not_found=True, allow_new=False, param_type=None)#

Extract parameter names and values from a configuration file.

The parameters which are extracted is dependent on whether the section, section_group and/or names variables are specified. A param_type dictionary can be defined to ensure variables are cast to the correct types.

Parameters:

  • config_file – Filename for input configuration file.

  • expected_param – Parameters within the configuration file which must be returned. An error will be raised if these parameters are not present.

  • section – Extract parameters from section name(s).

  • section_group – Extract parameters from all sections with this group (these groups). If section and section_group are both specified - section takes precedence.

  • names – Parameter names to extract (within section or section_group, if specified).

  • ignore_sections – Sections to ignore when reading in the configuration file (even if parameters are specified).

  • ignore_section_groups – Sections groups to ignore when reading in the configuration file (even if parameters are specified).

  • exclude_not_found – Whether to remove parameters which are not found in the input file or include them as None. Default = True.

  • allow_new – If a param_type is specified, whether to allow unrecognised parameters to be added without printing a warning. Default = False (i.e. Warning will be printed).

  • param_type – Nested dictionary of sections or groups and expected parameter names and types. See module header for expected formats of param_type dictionary.

Returns:

Parameter names and values.

Return type:

OrderedDict

openghg_inversions.config.config.find_param_key(param_type, section=None, section_group=None)#

The find_param_key function checks whether the keys within the param_type dictionary are for sections (e.g. ‘MCMC.MEASUREMENTS’) or groups (e.g. ‘MCMC’) and returns the relevant key(s). One of section or section_group should be specified. Returned key_type is one of ‘section’ or ‘section_group’.

Parameters:

  • param_type (dict) – Nested dictionary of expected parameter names and types. Key for each parameter dictionary can be the section heading or the overall group (e.g. for [MCMC.MEASUREMENTS], section group should be ‘MCMC’). See module header for expected formats of param_type dictionary.

  • section (str, optional) – Name of section in config file for the parameter

  • section_group (str, optional) – Name of group in config file for the parameter

Returns:

keys, key_type

Return type:

list,str

openghg_inversions.config.config.generate_from_template(template_file, output_file)#

Generate an example configuration file based on a template file.

Template files are normally used to inform the expected format of any input configuration file.

Parameters:

  • template_file – Input template file in expected .ini format.

  • output_file – Name of output file including path information.

Note

Writes output file. If output file is already present, the user will be asked whether the file should be overwritten. If the response is ‘N’ or ‘no’ or an unrecognised input an exception will be raised.

openghg_inversions.config.config.generate_param_dict(config_file)#

The generate_param_dict function creates a param_type nested dictionary from an input configuration file. This could be used on some fixed template config file to generate the parameter type dictionary which can then be applied to other configuration files of the same type.

Parameters:

config_file (str) – Filename for template configuration file (str).

Returns:

Parameter type dictionary from configuration file input with sections as keys for parameters and value types.

Return type:

nested OrderedDict

openghg_inversions.config.config.get_value(name, config, section, param_type=None)#

The get_value function extracts the value of a parameter from the configuration file. This value is then converted to the type specified within param_type (default from mcmc_param_type() function).

Parameters:

  • name (str) – Name of the parameter to extract

  • config (ConfigParser) – ConfigParser object created using the configparser class. Data from config file should have been read in.

  • section (str) – Name of section in config file for the parameter

  • param_type (dict, optional) – nested dictionary of parameter classes and expected parameter names and types. Key for each parameter dictionary can be the section heading or the overall classification (e.g. for [MCMC.MEASUREMENTS], classification should be ‘MCMC’). See module header for expected formats of param_type dictionary.

Returns:

value

If param_type is specified
If neither section nor section_group can be identified within param_type dictionary:

Exception raised and program exited

If parameter name cannot be identified within param_type dictionary:

Exception raised and program exited

openghg_inversions.config.config.list_check(string, force_convert=True, error=True)#

Convert input string to a list.

Parameters:

  • string – Value input from config file.

  • force_convert – Specifies whether conversion to a list should be forced (i.e. out = [out] if unable to evaluate any other way). Default = True.

  • error – Print error message if unable to evaluate.

Returns:

List if conversion successful, None if unable to convert to list.

Return type:

list or None

openghg_inversions.config.config.open_config(config_file)#

Open configuration files in the ini format.

Parameters:

config_file – Filename for input configuration file.

Returns:

Parsed configuration object.

Return type:

configparser.ConfigParser

openghg_inversions.config.config.str_check(string, error=True)#

The str_check function is used as part of checking the input from a configuration file. This function ensures the input remains as a string and removes any “ or ‘ characters.

Parameters:

  • string (str) – Value input from config file

  • error (bool, optional) – Print error message if unable to evaluate (bool).

Returns:

string (formatted)