caelus.utils – Basic utilities

Collection of low-level utilities that are accessed by other packages within CPL, and other code snippets that do not fit elsewhere within CPL. The modules present within utils package must only depend on external libraries or other modules within util, they must not import modules from other packages within CPL.

Struct	Dictionary that supports both key and attribute access.
osutils	Miscellaneous utilities

Struct Module

Implements Struct.

class caelus.utils.struct.Struct

Bases: OrderedDict, MutableMapping

Dictionary that supports both key and attribute access.

Struct is inspired by Matlab struct data structure that is intended to support both key and attribute access. It has the following features:

1. Preserves ordering of members as initialized

2. Provides attribute and dictionary-style lookups

3. Read/write YAML formatted data

yaml_decoder

alias of StructYAMLLoader

yaml_encoder

alias of StructYAMLDumper

classmethod from_yaml(stream)

Initialize mapping from a YAML string.

Parameters:

stream – A string or valid file handle

Returns:

YAML data as a python object

Return type:

Struct

classmethod load_yaml(filename)

Load a YAML file

Parameters:

filename (str) – Absolute path to YAML file

Returns:

YAML data as python object

Return type:

Struct

merge(*args)

Recursively update dictionary

Merge entries from maps provided such that new entries are added and existing entries are updated.

pget(path, sep='.')

Get value from a nested dictionary entry.

A convenience method that serves various purposes:

• Access values from a deeply nested dictionary if any of the keys are not valid python variable names.

• Return None if any of the intermediate keys are missing. Does not raise AttributeError.

By default, the method uses the . character to split keys similar to attribute access. However, this can be overridden by providing and extra sep argument.

Parameters:

• path (str) – The keys in individual dictionarys separated by sep

• sep (str) – Separator for splitting keys (default: “.”)

Returns:

Value corresponding to the key, or None if any of the keys

don’t exist.

pset(path, value, sep='.')

Set value for a nested dictionary entry.

A convenience method to set values in a nested mapping hierarchy without individually creating the intermediate dictionaries. Missing intermediate dictionaries will automatically be created with the same mapping class as the class of self.

Parameters:

• path (str) – The keys in individual dictionarys separated by sep

• value (object) – Object assigned to innermost key

• sep (str) – Separator for splitting keys (default: “.”)

Raises:

AttributeError – If the object assigned to is a non-mapping type and not the final key.

to_yaml(stream=None, default_flow_style=False, **kwargs)

Convert mapping to YAML format.

Parameters:

• stream (file) – A file handle where YAML is output

• default_flow_style (bool) –

  • False - pretty printing

  • True - No pretty printing

class caelus.utils.struct.StructMeta(name, bases, cdict)

Bases: ABCMeta

YAML interface registration

Simplify the registration of custom yaml loader/dumper classes for Struct class hierarchy.

caelus.utils.struct.gen_yaml_decoder(cls)

Generate a custom YAML decoder with non-default mapping class

Parameters:

cls – Class used for mapping

caelus.utils.struct.gen_yaml_encoder(cls)

Generate a custom YAML encoder with non-default mapping class

Parameters:

cls – Class used for mapping

caelus.utils.struct.merge(a, b, *args)

Recursively merge mappings and return consolidated dict.

Accepts a variable number of dictionary mappings and returns a new dictionary that contains the merged entries from all dictionaries. Note that the update occurs left to right, i.e., entries from later dictionaries overwrite entries from preceeding ones.

Returns:

The consolidated map

Return type:

dict

Miscellaneous utilities

This module implements functions that are utilized throughout CPL. They mostly provide a higher-level interface to various os.path functions to make it easier to perform some tasks.

set_work_dir	A with-block to execute code in a given directory.
ensure_directory	Check if directory exists, if not, create it.
abspath	Return the absolute path of the directory.
ostype	String indicating the operating system type
timestamp	Return a formatted timestamp for embedding in files

caelus.utils.osutils.abspath(pname)

Return the absolute path of the directory.

This function expands the user home directory as well as any shell variables found in the path provided and returns an absolute path.

Parameters:

pname (path) – Pathname to be expanded

Returns:

Absolute path after all substitutions

Return type:

path

caelus.utils.osutils.backup_file(fname, time_format=None, time_zone=<UTC>)

Given a filename return a timestamp based backup filename

Parameters:

• time_format – A time formatter suitable for strftime

• time_zone – Time zone used to generate timestamp (Default: UTC)

Returns:

A timestamped filename suitable for creating backups

Return type:

str

caelus.utils.osutils.clean_directory(dirname, preserve_patterns=None)

Utility function to remove files and directories from a given directory.

User can specify a list of filename patterns to preserve with the preserve_patterns argument. These patterns can contain shell wildcards to glob multiple files.

Parameters:

• dirname (path) – Absolute path to the directory whose entries are purged.

• preserve_patterns (list) – A list of shell wildcard patterns

caelus.utils.osutils.copy_tree(srcdir, destdir, symlinks=False, ignore_func=None)

Enchanced version of shutil.copytree

• removes the output directory if it already exists.

Parameters:

• srcdir (path) – path to source directory to be copied.

• destdir (path) – path (or new name) of destination directory.

• symlinks (bool) – as in shutil.copytree

• ignore_func (func) – as in shutil.copytree

caelus.utils.osutils.ensure_directory(dname)

Check if directory exists, if not, create it.

Parameters:

dname (path) – Directory name to check for

Returns:

Absolute path to the directory

Return type:

Path

caelus.utils.osutils.ostype()

String indicating the operating system type

Returns:

One of [“linux”, “darwin”, “windows”]

Return type:

str

caelus.utils.osutils.path_exists(pname)

Check path of the directory exists.

This function expands the user home directory as well as any shell variables found in the path provided and checks if that path exists.

Parameters:

pname (path) – Pathname to be checked

Returns:

True if path exists

Return type:

bool

caelus.utils.osutils.path_or_cwd(pdir: Path | str | None) → Path

Sanitize the given path. If None, return current workding directory

caelus.utils.osutils.remove_files_dirs(paths, basedir=None)

Remove files and/or directories

Parameters:

• paths (list) – A list of file paths to delete (no patterns allowed)

• basedir (path) – Base directory to search

caelus.utils.osutils.set_work_dir(dname, create=False)

A with-block to execute code in a given directory.

Parameters:

• dname (path) – Path to the working directory.

• create (bool) – If true, directory is created prior to execution

Returns:

Absolute path to the execution directory

Return type:

path

Example

>>> with osutils.set_work_dir("results_dir", create=True) as wdir:
...     with open(os.path.join(wdir, "results.dat"), 'w') as fh:
...         fh.write("Data")

caelus.utils.osutils.split_path(fname)

Split a path into directory, basename, extension

Returns:

(directory, basename, extension)

Return type:

tuple

caelus.utils.osutils.timestamp(time_format=None, time_zone=<UTC>)

Return a formatted timestamp for embedding in files

Parameters:

• time_format – A time formatter suitable for strftime

• time_zone – Time zone used to generate timestamp (Default: UTC)

Returns:

A formatted time string

Return type:

str

caelus.utils.osutils.user_home_dir()

Return the absolute path of the user’s home directory

caelus.utils.osutils.username()

Return the username of the current user