patroni.utils module

Utilitary objects and functions that can be used throughout Patroni code.

var tzutc

UTC time zone info object.

var logger

logger of this module.

var USER_AGENT

identifies the Patroni version, Python version, and the underlying platform.

var OCT_RE

regular expression to match octal numbers, signed or unsigned.

var DEC_RE

regular expression to match decimal numbers, signed or unsigned.

var HEX_RE

regular expression to match hex strings, signed or unsigned.

var DBL_RE

regular expression to match double precision numbers, signed or unsigned. Matches scientific notation too.

var WHITESPACE_RE

regular expression to match whitespace characters

class _patroni.utils.Retry(_max_tries: int | None = 1, delay: float = 0.1, backoff: int = 2, max_jitter: float = 0.8, max_delay: int = 3600, sleep_func: ~typing.Callable[[int | float], None] = <function _sleep>, deadline: int | float | None = None, retry_exceptions: ~typing.Type[Exception] | typing.Tuple[typing.Type[Exception], …​] = <class 'patroni.exceptions.PatroniException'>) View on GitHub

Bases: object + Helper for retrying a method in the face of retryable exceptions. +

Variables:

+

__init\\__(max_tries: int | None = 1, delay: float = 0.1, backoff: int = 2, max_jitter: float = 0.8, max_delay: int = 3600, sleep_func: ~typing.Callable[[int | float], None] = <function _sleep>, deadline: int | float | None = None, retry_exceptions: ~typing.Type[Exception] | typing.Tuple[typing.Type[Exception], …​] = <class 'patroni.exceptions.PatroniException'>) → None View on GitHub

Create a Retry instance for retrying function calls. +

Parameters

+

copy() → Retry View on GitHub

Return a clone of this retry manager. +

ensure_deadline(timeout: float, raise_ex: Exception | None = None) → bool View on GitHub

Calculates, sets, and checks the remaining deadline time. +

Parameters
Returns

False if deadline is smaller than a provided timeout and raise_ex isn’t set. Otherwise True.

Raises

Exception: raise_ex if calculated deadline is smaller than provided timeout. +

reset() → None View on GitHub

Reset the attempt counter, delay and stop time. +

property _sleeptime: float_

Get next cycle sleep time.
It is based on the current delay plus a number up to max_jitter. +

property _stoptime: float_

Get the current stop time. +

update_delay() → None View on GitHub

Set next cycle delay. + It will be the minimum value between: + __\\__ __\\__

exception _patroni.utils.RetryFailedError(_value: Any) View on GitHub

Bases: PatroniException + Maximum number of attempts exhausted in retry operation.

patroni.utils.sleep(_interval: int | float) → None View on GitHub

Wrap sleep(). +

Parameters:

interval – Delay execution for a given number of seconds. The argument may be a floating point number for subsecond precision.

patroni.utils.cluster_as_json(cluster: Cluster, global_config: GlobalConfig | None = None) → Dict[ str, Any] View on GitHub

Get a JSON representation of cluster. +

Parameters:
Returns:

JSON representation of cluster. + These are the possible keys in the returning object depending on the available information in cluster: + __\\__

  • members: list of members in the cluster. Each value is a dict that may have the following keys: + __\\__ __\\__

  • pause: True if cluster is in maintenance mode;

  • scheduled_switchover: if a switchover has been scheduled, then it contains this entry with these keys: + __\\__ __\\__ __\\__

    patroni.utils.compare_values(vartype: str, unit: str | None, old_value: Any, new_value: Any) → bool View on GitHub

    Check if old_value and new_value are equivalent after parsing them as vartype. +

Parameters:

unit – base unit to be used as argument when calling parse_int() or parse_real() for new_value. + old_value – value to be compared with new_value. + new_value – value to be compared with old_value.

Returns:

True if old_value is equivalent to new_value when both are parsed as vartype.

Example:
>>> compare_values('enum', None, 'remote_write', 'REMOTE_WRITE')
True
+
>>> compare_values('string', None, 'remote_write', 'REMOTE_WRITE')
False
+
>>> compare_values('real', None, '1e-06', 0.000001)
True
+
>>> compare_values('integer', 'MB', '6GB', '6GB')
False
+
>>> compare_values('integer', None, '6GB', '6GB')
False
+
>>> compare_values('integer', '16384kB', '64', ' 0x400 MB ')
True
+
>>> compare_values('integer', '2MB', 524288, '1TB')
True
+
>>> compare_values('integer', 'MB', 1048576, '1TB')
True
+
>>> compare_values('integer', 'kB', 4098, '4097.5kB')
True
patroni.utils.convert_to_base_unit(value: int | float, unit: str, base_unit: str | None) → int | float | None View on GitHub

Convert value as a unit of compute information or time to base_unit. +

Parameters:

base_unit – + target unit in the conversion. May contain the target unit with an associated value, e.g 512MB. Accepts these units (case sensitive): + __\\__

  • For space: B, kB, or MB;

  • For time: ms, s, or min. __\\__

Returns:

value in unit converted to base_unit. Returns None if unit or base_unit is invalid.

Example:
>>> convert_to_base_unit(1, 'GB', '256MB')
4
+
>>> convert_to_base_unit(1, 'GB', 'MB')
1024
+
>>> convert_to_base_unit(1, 'gB', '512MB') is None
True
+
>>> convert_to_base_unit(1, 'GB', '512 MB') is None
True
patroni.utils.data_directory_is_empty(data_dir: str) → bool View on GitHub

Check if a PostgreSQL data directory is empty. + Note

In non-Windows environments _data_dir_ is also considered empty if it only contains hidden files and/or `+lost+found+` directory.
+
Parameters:;;
  *data_dir* – the PostgreSQL data directory to be checked.
Returns:;;
  `+True+` if _data_dir_ is empty.
patroni.utils.deep_compare(obj1: Dict[ Any, Any | Dict[ Any, Any]], obj2: Dict[ Any, Any | Dict[ Any, Any]]) → bool View on GitHub

Recursively compare two dictionaries to check if they are equal in terms of keys and values. + Note

Values are compared based on their string representation.
+
Parameters:;;
Returns:;;
  `+True+` if all keys and values match between the two dictionaries.
Example:;;
>>> deep_compare({'1': None}, {})
False

+

>>> deep_compare({'1': {}}, {'1': None})
False

+

>>> deep_compare({'1': [1]}, {'1': [2]})
False

+

>>> deep_compare({'1': 2}, {'1': '2'})
True

+

>>> deep_compare({'1': {'2': [3, 4]}}, {'1': {'2': [3, 4]}})
True
patroni.utils.enable_keepalive(sock: socket, timeout: int, idle: int, cnt: int = 3) → None View on GitHub

Enable keepalive for sock. + Will set socket options depending on the platform, as per return of keepalive_socket_options(). + Note

Value for `+TCP_KEEPINTVL+` will be calculated through link:#patroni.utils.keepalive_intvl[`+keepalive_intvl()+`] based on _timeout_, _idle_, and _cnt_.
+
Parameters:;;
Returns:;;
  output of `+ioctl()+` if we are on Windows, nothing otherwise.
patroni.utils.get_major_version(bin_dir: str | None = None, bin_name: str = 'postgres') → str View on GitHub

Get the major version of PostgreSQL. + It is based on the output of postgres`+ ``--version+`. +

Parameters:
Returns:

the PostgreSQL major version.

Raises:

PatroniException: if the postgres binary call failed due to OSError.

Example:
patroni.utils.is_subpath(d1: str, d2: str) → bool View on GitHub

Check if the file system path d2 is contained within d1 after resolving symbolic links. + Note

It will not check if the paths actually exist, it will only expand the paths and resolve any symbolic links that happen to be found.
+
Parameters:;;
Returns:;;
  `+True+` if _d1_ is a subpath of _d2_.
patroni.utils.iter_response_objects(response: HTTPResponse) → Iterator[ Dict[ str, Any]] View on GitHub

Iterate over the chunks of a HTTPResponse and yield each JSON document that is found. +

Parameters:

response – the HTTP response from which JSON documents will be retrieved.

Yields:

current JSON document.

patroni.utils.keepalive_intvl(timeout: int, idle: int, cnt: int = 3) → int View on GitHub

Calculate the value to be used as TCP_KEEPINTVL based on timeout, idle, and cnt. +

Parameters:
Returns:

the value to be used as TCP_KEEPINTVL.

patroni.utils.keepalive_socket_options(timeout: int, idle: int, cnt: int = 3) → Iterator[ Tuple[ int, int, int]] View on GitHub

Get all keepalive related options to be set in a socket. +

Parameters:
Yields:

all keepalive related socket options to be set. The first item in the tuple is the protocol, the second item is the option, and the third item is the value to be used. The return values depend on the platform: + __\\__

Linux
  • SO_KEEPALIVE;

  • TCP_USER_TIMEOUT;

  • TCP_KEEPIDLE;

  • TCP_KEEPINTVL;

  • TCP_KEEPCNT.

MacOS
  • SO_KEEPALIVE;

  • TCP_KEEPIDLE;

  • TCP_KEEPINTVL;

  • TCP_KEEPCNT. __\\__

patroni.utils.parse_bool(value: Any) → bool | None View on GitHub

Parse a given value to a bool object. + Note

The parsing is case-insensitive, and takes into consideration these values:

+

Parameters:

value – value to be parsed to bool.

Returns:

the parsed value. If not able to parse, returns None.

Example:
>>> parse_bool(1)
True
+
>>> parse_bool('off')
False
+
>>> parse_bool('foo')
patroni.utils.parse_int(value: Any, base_unit: str | None = None) → int | None View on GitHub

Parse value as an int. +

Parameters:
Returns:

the parsed value, if able to parse. Otherwise returns None.

Example:
>>> parse_int('1') == 1
True
+
>>> parse_int(' 0x400 MB ', '16384kB') == 64
True
+
>>> parse_int('1MB', 'kB') == 1024
True
+
>>> parse_int('1000 ms', 's') == 1
True
+
>>> parse_int('1TB', 'GB') is None
True
+
>>> parse_int(50, None) == 50
True
+
>>> parse_int("51", None) == 51
True
+
>>> parse_int("nonsense", None) == None
True
+
>>> parse_int("nonsense", "kB") == None
True
+
>>> parse_int("nonsense") == None
True
+
>>> parse_int(0) == 0
True
+
>>> parse_int('6GB', '16MB') == 384
True
+
>>> parse_int('4097.4kB', 'kB') == 4097
True
+
>>> parse_int('4097.5kB', 'kB') == 4098
True
patroni.utils.parse_real(value: Any, base_unit: str | None = None) → float | None View on GitHub

Parse value as a float. +

Parameters:
Returns:

the parsed value, if able to parse. Otherwise returns None.

Example:
>>> parse_real(' +0.0005 ') == 0.0005
True
+
>>> parse_real('0.0005ms', 'ms') == 0.0
True
+
>>> parse_real('0.00051ms', 'ms') == 0.001
True
patroni.utils.patch_config(config: Dict[ Any, Any | Dict[ Any, Any]], data: Dict[ Any, Any | Dict[ Any, Any]]) → bool View on GitHub

Update and append to dictionary config from overrides in data. + Note +

Parameters:
Returns:

True if config was changed.

patroni.utils.polling_loop(timeout: int | float, interval: int | float = 1) → Iterator[ int] View on GitHub

Return an iterator that returns values every interval seconds until timeout has passed. + Note

Timeout is measured from start of iteration.
+
Parameters:;;
Yields:;;
  current iteration counter, starting from `+0+`.
patroni.utils.read_stripped(file_path: str) → Iterator[ str] View on GitHub

Iterate over stripped lines in the given file. +

Parameters:

file_path – path to the file to read from

Yields:

each line from the given file stripped

patroni.utils.split_host_port(value: str, default_port: int | None) → Tuple[ str, int] View on GitHub

Extract host(s) and port from value. +

Parameters:

Each host portion of value can be either: + __\\__

  • A FQDN; or

  • An IPv4 address; or

  • An IPv6 address, with or without square brackets. __\\__ + default_port – if no port can be found in param, use default_port instead.

Returns:

the first item is composed of a CSV list of hosts from value, and the second item is either the port from value or default_port.

Example:
>>> split_host_port('127.0.0.1', 5432)
('127.0.0.1', 5432)
+
>>> split_host_port('127.0.0.1:5400', 5432)
('127.0.0.1', 5400)
+
>>> split_host_port('127.0.0.1,192.168.0.101:5400', 5432)
('127.0.0.1,192.168.0.101', 5400)
+
>>> split_host_port('127.0.0.1,www.mydomain.com,[fe80:0:0:0:213:72ff:fe3c:21bf], 0:0:0:0:0:0:0:0:5400', 5432)
('127.0.0.1,www.mydomain.com,fe80:0:0:0:213:72ff:fe3c:21bf,0:0:0:0:0:0:0:0', 5400)
patroni.utils.strtod(value: Any) → Tuple[ float | None, str] View on GitHub

Extract the double precision part from the beginning of a string that reprensents a configuration value. + As most as possible close equivalent of strtod(3) C function, which is used by postgres to parse parameter values. +

Parameters:

value – any value from which we want to extract a double precision.

Returns:

the first item is the extracted double precision from value, and the second item is the remaining string of value. If not able to match a double precision in value, then the first item will be None, and the second item will be the original value.

Example:
>>> strtod(' A ') == (None, 'A')
True
+
>>> strtod('1 A ') == (1.0, ' A')
True
+
>>> strtod('1.5A') == (1.5, 'A')
True
+
>>> strtod('8.325e-10A B C') == (8.325e-10, 'A B C')
True
patroni.utils.strtol(value: Any, strict: bool | None = True) → Tuple[ int | None, str] View on GitHub

Extract the long integer part from the beginning of a string that represents a configuration value. + As most as possible close equivalent of strtol(3) C function (with base=0), which is used by postgres to parse parameter values. + Takes into consideration numbers represented either as hex, octal or decimal formats. +

Parameters:
Returns:

the first item is the extracted long integer from value, and the second item is the remaining string of value. If not able to match a long integer in value, then the first item will be either None or 1 (depending on strict argument), and the second item will be the original value.

Example:
>>> strtol(0) == (0, '')
True
+
>>> strtol(1) == (1, '')
True
+
>>> strtol(9) == (9, '')
True
+
>>> strtol(' +0x400MB') == (1024, 'MB')
True
+
>>> strtol(' -070d') == (-56, 'd')
True
+
>>> strtol(' d ') == (None, 'd')
True
+
>>> strtol(' 1 d ') == (1, ' d')
True
+
>>> strtol('9s', False) == (9, 's')
True
+
>>> strtol(' s ', False) == (1, 's')
True
patroni.utils.unquote(string: str) → str View on GitHub

Unquote a fully quoted string. +

Parameters:

string – The string to be checked for quoting.

Returns:

The string with quotes removed, if it is a fully quoted single string, or the original string if quoting is not detected, or unquoting was not possible.

Examples:

A string with quotes will have those quotes removed +

>>> unquote('"a quoted string"')
'a quoted string'
+
A _string_ with multiple quotes will be returned as is
+
>>> unquote('"a multi" "quoted string"')
'"a multi" "quoted string"'
+
So will a _string_ with unbalanced quotes
+
>>> unquote('unbalanced "quoted string')
'unbalanced "quoted string'
patroni.utils.uri(proto: str, netloc: List[ str] | Tuple[ str, int | str] | str, path: str | None = '', user: str | None = None) → str View on GitHub

Construct URI from given arguments. +

Parameters:

+ A str in either of these formats: + __\\__ __\\__ + In all cases, each host portion of netloc can be either: + __\\__

  • An FQDN; or

  • An IPv4 address; or

  • An IPv6 address, with or without square brackets. __\\__ + path – the URI path. + user – the authenticating user, if any.

    Returns

    constructed URI.

patroni.utils.validate_directory(d: str, msg: str = '\{} \{}') → None View on GitHub

Ensure directory exists and is writable. + Note

If the directory does not exist, link:#patroni.utils.validate_directory[`+validate_directory()+`] will attempt to create it.
+
Parameters:;;
Raises:;;
  link:patroni.exceptions.html#patroni.exceptions.PatroniException[`+PatroniException+`]: if any issue is observed while validating _d_. Can be thrown if:
  +
  \\__\\__
  \\__\\__

© Copyright 2015 Compose, Zalando SE. Revision 3d527f57.

Built with Sphinx using a theme provided by Read the Docs.

Read the Docs v: master