Untitled :: Tecnisys Docs

patroni.utils module

Utilitary objects and functions that can be used throughout Patroni code.

var tzutc

UTC time zone info object.

var logger

logger of this module.

var USER_AGENT

identifies the Patroni version, Python version, and the underlying platform.

var OCT_RE

regular expression to match octal numbers, signed or unsigned.

var DEC_RE

regular expression to match decimal numbers, signed or unsigned.

var HEX_RE

regular expression to match hex strings, signed or unsigned.

var DBL_RE

regular expression to match double precision numbers, signed or unsigned. Matches scientific notation too.

var WHITESPACE_RE

regular expression to match whitespace characters

class _patroni.utils.Retry(_max_tries: int | None = 1, delay: float = 0.1, backoff: int = 2, max_jitter: float = 0.8, max_delay: int = 3600, sleep_func: ~typing.Callable[[int | float], None] = <function _sleep>, deadline: int | float | None = None, retry_exceptions: ~typing.Type[Exception] | _{typing.Tuple[}typing.Type[Exception], …] = <class 'patroni.exceptions.PatroniException'>) View on GitHub 

Bases: object + Helper for retrying a method in the face of retryable exceptions. +

Variables:: +
__init\\__(max_tries: int | None = 1, delay: float = 0.1, backoff: int = 2, max_jitter: float = 0.8, max_delay: int = 3600, sleep_func: ~typing.Callable[[int | float], None] = <function _sleep>, deadline: int | float | None = None, retry_exceptions: ~typing.Type[Exception] | _{typing.Tuple[}typing.Type[Exception], …] = <class 'patroni.exceptions.PatroniException'>) → None View on GitHub : Create a Retry instance for retrying function calls. +

Parameters

copy() → Retry View on GitHub : Return a clone of this retry manager. +
ensure_deadline(timeout: float, raise_ex: Exception | None = None) → bool View on GitHub : Calculates and checks the remaining deadline time. +

Parameters

Returns

False if deadline is smaller than a provided timeout and raise_ex isn’t set. Otherwise True.

Raises

Exception: raise_ex if calculated deadline is smaller than provided timeout. +

reset() → None View on GitHub 

Reset the attempt counter, delay and stop time. +

property _sleeptime: float_

Get next cycle sleep time.
It is based on the current delay plus a number up to max_jitter. +

property _stoptime: float_

Get the current stop time. +

update_delay() → None View on GitHub 

Set next cycle delay. + It will be the minimum value between: + __\\__ __\\__

exception _patroni.utils.RetryFailedError(_value: Any) View on GitHub : Bases: PatroniException + Maximum number of attempts exhausted in retry operation.
patroni.utils.sleep(_interval: int | float) → None View on GitHub : Wrap sleep(). +

Parameters:

interval – Delay execution for a given number of seconds. The argument may be a floating point number for subsecond precision.

patroni.utils.cluster_as_json(cluster: Cluster) → Dict[ str, Any] View on GitHub : Get a JSON representation of cluster. +

Parameters:

cluster – the Cluster object to be parsed as JSON.

Returns:

JSON representation of cluster. + These are the possible keys in the returning object depending on the available information in cluster: + __\\__

members: list of members in the cluster. Each value is a dict that may have the following keys: + __\\__ __\\__
pause: True if cluster is in maintenance mode;
scheduled_switchover: if a switchover has been scheduled, then it contains this entry with these keys: + __\\__ __\\__ __\\__

patroni.utils.compare_values(vartype: str, unit: str | None, settings_value: Any, config_value: Any) → bool View on GitHub 

Check if the value from pg_settings and from Patroni config are equivalent after parsing them as vartype. +

Parameters:

unit – base unit to be used as argument when calling parse_int() or parse_real() for config_value. + settings_value – value to be compared with config_value. + config_value – value to be compared with settings_value.

Returns:

True if settings_value is equivalent to config_value when both are parsed as vartype.

Example:

>>> compare_values('enum', None, 'remote_write', 'REMOTE_WRITE')
True

>>> compare_values('string', None, 'remote_write', 'REMOTE_WRITE')
False

>>> compare_values('real', None, '1e-06', 0.000001)
True

>>> compare_values('integer', 'MB', '6GB', '6GB')
False

>>> compare_values('integer', None, '6GB', '6GB')
False

>>> compare_values('integer', '16384kB', '64', ' 0x400 MB ')
True

>>> compare_values('integer', '2MB', 524288, '1TB')
True

>>> compare_values('integer', 'MB', 1048576, '1TB')
True

>>> compare_values('integer', 'kB', 4098, '4097.5kB')
True

patroni.utils.convert_int_from_base_unit(base_value: int, base_unit: str | None) → str | None View on GitHub 

Convert an integer value in some base unit to a human-friendly unit. + The output unit is chosen so that it’s the greatest unit that can represent the value without loss. +

Parameters:
Returns:: str value representing base_value converted from base_unit to the greatest possible human-friendly unit, or None if conversion failed.
Example:

>>> convert_int_from_base_unit(1024, 'kB')
'1MB'

>>> convert_int_from_base_unit(1025, 'kB')
'1025kB'

>>> convert_int_from_base_unit(4, '256MB')
'1GB'

>>> convert_int_from_base_unit(4, '256 MB') is None
True

>>> convert_int_from_base_unit(1024, 'KB') is None
True

patroni.utils.convert_real_from_base_unit(base_value: float, base_unit: str | None) → str | None View on GitHub 

Convert an floating-point value in some base unit to a human-friendly unit. + Same as convert_int_from_base_unit(), except we have to do the math a bit differently, and there’s a possibility that we don’t find any exact divisor. +

Parameters:
Returns:: str value representing base_value converted from base_unit to the greatest possible human-friendly unit, or None if conversion failed.
Example:

>>> convert_real_from_base_unit(5, 'ms')
'5ms'

>>> convert_real_from_base_unit(2.5, 'ms')
'2500us'

>>> convert_real_from_base_unit(4.0, '256MB')
'1GB'

>>> convert_real_from_base_unit(4.0, '256 MB') is None
True

patroni.utils.convert_to_base_unit(value: int | float, unit: str, base_unit: str | None) → int | float | None View on GitHub 

Convert value as a unit of compute information or time to base_unit. +

Parameters:

base_unit – + target unit in the conversion. May contain the target unit with an associated value, e.g 512MB. Accepts these units (case sensitive): + __\\__

For space: B, kB, or MB;
For time: ms, s, or min. __\\__

Returns:

value in unit converted to base_unit. Returns None if unit or base_unit is invalid.

Example:

>>> convert_to_base_unit(1, 'GB', '256MB')
4

>>> convert_to_base_unit(1, 'GB', 'MB')
1024

>>> convert_to_base_unit(1, 'gB', '512MB') is None
True

>>> convert_to_base_unit(1, 'GB', '512 MB') is None
True

patroni.utils.data_directory_is_empty(data_dir: str) → bool View on GitHub 

Check if a PostgreSQL data directory is empty. + Note

In non-Windows environments _data_dir_ is also considered empty if it only contains hidden files and/or `+lost+found+` directory.
+
Parameters:;;
  *data_dir* – the PostgreSQL data directory to be checked.
Returns:;;
  `+True+` if _data_dir_ is empty.

patroni.utils.deep_compare(obj1: Dict[ Any, Any | Dict[ Any, Any]], obj2: Dict[ Any, Any | Dict[ Any, Any]]) → bool View on GitHub 

Recursively compare two dictionaries to check if they are equal in terms of keys and values. + Note

Values are compared based on their string representation.
+
Parameters:;;
Returns:;;
  `+True+` if all keys and values match between the two dictionaries.
Example:;;

>>> deep_compare({'1': None}, {})
False

>>> deep_compare({'1': {}}, {'1': None})
False

>>> deep_compare({'1': [1]}, {'1': [2]})
False

>>> deep_compare({'1': 2}, {'1': '2'})
True

>>> deep_compare({'1': {'2': [3, 4]}}, {'1': {'2': [3, 4]}})
True

patroni.utils.enable_keepalive(sock: socket, timeout: int, idle: int, cnt: int = 3) → None View on GitHub 

Enable keepalive for sock. + Will set socket options depending on the platform, as per return of keepalive_socket_options(). + Note

Value for `+TCP_KEEPINTVL+` will be calculated through link:#patroni.utils.keepalive_intvl[`+keepalive_intvl()+`] based on _timeout_, _idle_, and _cnt_.
+
Parameters:;;
Returns:;;
  output of `+ioctl()+` if we are on Windows, nothing otherwise.

patroni.utils.get_conversion_table(base_unit: str) → Dict[ str, Dict[ str, int | float]] View on GitHub 

Get conversion table for the specified base unit. + If no conversion table exists for the passed unit, return an empty OrderedDict. +

Parameters:: base_unit – unit to choose the conversion table for.
Returns:: OrderedDict object.

patroni.utils.get_major_version(bin_dir: str | None = None, bin_name: str = 'postgres') → str View on GitHub 

Get the major version of PostgreSQL. + It is based on the output of postgres`+ ``--version+`. +

Parameters:
Returns:: the PostgreSQL major version.
Raises:: PatroniException: if the postgres binary call failed due to OSError.
Example:

patroni.utils.is_subpath(d1: str, d2: str) → bool View on GitHub 

Check if the file system path d2 is contained within d1 after resolving symbolic links. + Note

It will not check if the paths actually exist, it will only expand the paths and resolve any symbolic links that happen to be found.
+
Parameters:;;
Returns:;;
  `+True+` if _d1_ is a subpath of _d2_.

patroni.utils.iter_response_objects(response: HTTPResponse) → Iterator[ Dict[ str, Any]] View on GitHub 

Iterate over the chunks of a HTTPResponse and yield each JSON document that is found. +

Parameters:: response – the HTTP response from which JSON documents will be retrieved.
Yields:: current JSON document.

patroni.utils.keepalive_intvl(timeout: int, idle: int, cnt: int = 3) → int View on GitHub 

Calculate the value to be used as TCP_KEEPINTVL based on timeout, idle, and cnt. +

Parameters:
Returns:: the value to be used as TCP_KEEPINTVL.

patroni.utils.keepalive_socket_options(timeout: int, idle: int, cnt: int = 3) → Iterator[ Tuple[ int, int, int]] View on GitHub 

Get all keepalive related options to be set in a socket. +

Parameters:

Yields:

all keepalive related socket options to be set. The first item in the tuple is the protocol, the second item is the option, and the third item is the value to be used. The return values depend on the platform: + __\\__

Linux

SO_KEEPALIVE;
TCP_USER_TIMEOUT;
TCP_KEEPIDLE;
TCP_KEEPINTVL;
TCP_KEEPCNT.

MacOS

SO_KEEPALIVE;
TCP_KEEPIDLE;
TCP_KEEPINTVL;
TCP_KEEPCNT. __\\__

patroni.utils.maybe_convert_from_base_unit(base_value: str, vartype: str, base_unit: str | None) → str View on GitHub 

Try to convert integer or real value in a base unit to a human-readable unit. + Value is passed as a string. If parsing or subsequent conversion fails, the original value is returned. +

Parameters:
Returns:: str value representing base_value converted from base_unit to the greatest possible human-friendly unit, or base_value string if conversion failed.
Example:

>>> maybe_convert_from_base_unit('5', 'integer', 'ms')
'5ms'

>>> maybe_convert_from_base_unit('4.2', 'real', 'ms')
'4200us'

>>> maybe_convert_from_base_unit('on', 'bool', None)
'on'

>>> maybe_convert_from_base_unit('', 'integer', '256MB')
''

patroni.utils.parse_bool(value: Any) → bool | None View on GitHub 

Parse a given value to a bool object. + Note

The parsing is case-insensitive, and takes into consideration these values:: +
Parameters:: value – value to be parsed to bool.
Returns:: the parsed value. If not able to parse, returns None.
Example:

>>> parse_bool(1)
True

>>> parse_bool('off')
False

>>> parse_bool('foo')

patroni.utils.parse_int(value: Any, base_unit: str | None = None) → int | None View on GitHub 

Parse value as an int. +

Parameters:
Returns:: the parsed value, if able to parse. Otherwise returns None.
Example:

>>> parse_int('1') == 1
True

>>> parse_int(' 0x400 MB ', '16384kB') == 64
True

>>> parse_int('1MB', 'kB') == 1024
True

>>> parse_int('1000 ms', 's') == 1
True

>>> parse_int('1TB', 'GB') is None
True

>>> parse_int(50, None) == 50
True

>>> parse_int("51", None) == 51
True

>>> parse_int("nonsense", None) == None
True

>>> parse_int("nonsense", "kB") == None
True

>>> parse_int("nonsense") == None
True

>>> parse_int(0) == 0
True

>>> parse_int('6GB', '16MB') == 384
True

>>> parse_int('4097.4kB', 'kB') == 4097
True

>>> parse_int('4097.5kB', 'kB') == 4098
True

patroni.utils.parse_real(value: Any, base_unit: str | None = None) → float | None View on GitHub 

Parse value as a float. +

Parameters:
Returns:: the parsed value, if able to parse. Otherwise returns None.
Example:

>>> parse_real(' +0.0005 ') == 0.0005
True

>>> parse_real('0.0005ms', 'ms') == 0.0
True

>>> parse_real('0.00051ms', 'ms') == 0.001
True

patroni.utils.patch_config(config: Dict[ Any, Any | Dict[ Any, Any]], data: Dict[ Any, Any | Dict[ Any, Any]]) → bool View on GitHub 

Update and append to dictionary config from overrides in data. + Note +

Parameters:
Returns:: True if config was changed.

patroni.utils.polling_loop(timeout: int | float, interval: int | float = 1) → Iterator[ int] View on GitHub 

Return an iterator that returns values every interval seconds until timeout has passed. + Note

Timeout is measured from start of iteration.
+
Parameters:;;
Yields:;;
  current iteration counter, starting from `+0+`.

patroni.utils.read_stripped(file_path: str) → Iterator[ str] View on GitHub 

Iterate over stripped lines in the given file. +

Parameters:: file_path – path to the file to read from
Yields:: each line from the given file stripped

patroni.utils.split_host_port(value: str, default_port: int | None) → Tuple[ str, int] View on GitHub 

Extract host(s) and port from value. +

Parameters:

Each host portion of value can be either: + __\\__

A FQDN; or
An IPv4 address; or
An IPv6 address, with or without square brackets. __\\__ + default_port – if no port can be found in param, use default_port instead.

Returns:

the first item is composed of a CSV list of hosts from value, and the second item is either the port from value or default_port.

Example:

>>> split_host_port('127.0.0.1', 5432)
('127.0.0.1', 5432)

>>> split_host_port('127.0.0.1:5400', 5432)
('127.0.0.1', 5400)

>>> split_host_port('127.0.0.1,192.168.0.101:5400', 5432)
('127.0.0.1,192.168.0.101', 5400)

>>> split_host_port('127.0.0.1,www.mydomain.com,[fe80:0:0:0:213:72ff:fe3c:21bf], 0:0:0:0:0:0:0:0:5400', 5432)
('127.0.0.1,www.mydomain.com,fe80:0:0:0:213:72ff:fe3c:21bf,0:0:0:0:0:0:0:0', 5400)

patroni.utils.strtod(value: Any) → Tuple[ float | None, str] View on GitHub 

Extract the double precision part from the beginning of a string that reprensents a configuration value. + As most as possible close equivalent of strtod(3) C function, which is used by postgres to parse parameter values. +

Parameters:: value – any value from which we want to extract a double precision.
Returns:: the first item is the extracted double precision from value, and the second item is the remaining string of value. If not able to match a double precision in value, then the first item will be None, and the second item will be the original value.
Example:

>>> strtod(' A ') == (None, 'A')
True

>>> strtod('1 A ') == (1.0, ' A')
True

>>> strtod('1.5A') == (1.5, 'A')
True

>>> strtod('8.325e-10A B C') == (8.325e-10, 'A B C')
True

patroni.utils.strtol(value: Any, strict: bool | None = True) → Tuple[ int | None, str] View on GitHub 

Extract the long integer part from the beginning of a string that represents a configuration value. + As most as possible close equivalent of strtol(3) C function (with base=0), which is used by postgres to parse parameter values. + Takes into consideration numbers represented either as hex, octal or decimal formats. +

Parameters:
Returns:: the first item is the extracted long integer from value, and the second item is the remaining string of value. If not able to match a long integer in value, then the first item will be either None or 1 (depending on strict argument), and the second item will be the original value.
Example:

>>> strtol(0) == (0, '')
True

>>> strtol(1) == (1, '')
True

>>> strtol(9) == (9, '')
True

>>> strtol(' +0x400MB') == (1024, 'MB')
True

>>> strtol(' -070d') == (-56, 'd')
True

>>> strtol(' d ') == (None, 'd')
True

>>> strtol(' 1 d ') == (1, ' d')
True

>>> strtol('9s', False) == (9, 's')
True

>>> strtol(' s ', False) == (1, 's')
True

patroni.utils.unquote(string: str) → str View on GitHub 

Unquote a fully quoted string. +

Parameters:: string – The string to be checked for quoting.
Returns:: The string with quotes removed, if it is a fully quoted single string, or the original string if quoting is not detected, or unquoting was not possible.
Examples:: A string with quotes will have those quotes removed +

>>> unquote('"a quoted string"')
'a quoted string'

+
A _string_ with multiple quotes will be returned as is
+

>>> unquote('"a multi" "quoted string"')
'"a multi" "quoted string"'

+
So will a _string_ with unbalanced quotes
+

>>> unquote('unbalanced "quoted string')
'unbalanced "quoted string'

Construct URI from given arguments. +

Parameters:

+ A str in either of these formats: + __\\__ __\\__ + In all cases, each host portion of netloc can be either: + __\\__

An FQDN; or
An IPv4 address; or
An IPv6 address, with or without square brackets. __\\__ + path – the URI path. + user – the authenticating user, if any.

Returns

constructed URI.

patroni.utils.validate_directory(d: str, msg: str = '\{} \{}') → None View on GitHub 

Ensure directory exists and is writable. + Note

If the directory does not exist, link:#patroni.utils.validate_directory[`+validate_directory()+`] will attempt to create it.
+
Parameters:;;
Raises:;;
  link:patroni.exceptions.html#patroni.exceptions.PatroniException[`+PatroniException+`]: if any issue is observed while validating _d_. Can be thrown if:
  +
  \\__\\__
  \\__\\__

Previous Next

Built with Sphinx using a theme provided by Read the Docs.

Read the Docs v: latest

bugfix-readthedocs

Project Home

+ Builds