bexhoma.configurations module

DBMS configuration subpackage for bexhoma.

Provides SutConfiguration (the primary configuration class) together with helper classes that implement specialised subsystems via composition.

The module-level name default is kept as an alias so that any code that still imports from bexhoma.configurations import default continues to work without changes.

The module-level YAML helpers and the load_data_asynch thread function are re-exported from manifest and loading respectively.

class bexhoma.configurations.BenchmarkRunner(config: SutConfiguration)

Bases: object

Creates and monitors benchmarker Kubernetes jobs for a configuration.

Parameters:: config (SutConfiguration) – The parent configuration this runner belongs to.

run_pod(connection: str | None = None, alias: str = '', dialect: str = '', query: str | None = None, app: str = '', component: str = 'benchmarker', experiment: str = '', configuration: str = '', client: str = '1', parallelism: int = 1, only_prepare: bool = False, benchmark_run: str = '', template_override: str = '') → None

Start a benchmarker job pod, upload configs, and optionally wait for it.

Sets metadata in the connection config, copies query.config and connection.config to the first pod of the job (result folder is mounted into every pod), then optionally submits the job.

Parameters:

connection – dbmsbenchmarker connection name; defaults to self._config.configuration.
alias – Display alias to anonymise the DBMS name.
dialect – SQL dialect override string.
query – Fix the benchmark to a single query; None means all.
app – App label override; defaults to self._config.appname.
component – Component label (default 'benchmarker').
experiment – Experiment code (defaults to self._config.code).
configuration – DBMS configuration name; defaults to connection.
client – Sequential client-round index string.
parallelism – Number of parallel benchmarker pods.
only_prepare – When True, upload configs but do not submit the job.
benchmark_run – 1-based parallel benchmark index within one client round.
template_override – When non-empty, overrides the default YAML job template.

class bexhoma.configurations.ComponentStatus(config: SutConfiguration)

Bases: object

Encapsulates all Kubernetes pod-state predicates for a configuration.

Replaces the sut_is_*, monitoring_is_*, and maintaining_is_* methods that were formerly on SutConfiguration.

Parameters:: config (SutConfiguration) – The parent configuration this status object belongs to.

maintaining_pending() → bool

Return True iff any maintaining pod is in Pending state.

Returns:: True if a maintaining pod is pending.
Return type:: bool

maintaining_running() → bool

Return True iff maintaining pods equal the target count (running or succeeded).

Returns:: True if maintaining job has reached completion.
Return type:: bool

monitoring_pending() → bool

Return True iff the monitoring pod is in Pending state.

Returns:: True if monitoring pod is pending.
Return type:: bool

monitoring_running() → bool

Return True iff the monitoring deployment pod is Running.

Returns:: True if monitoring is active and running.
Return type:: bool

sut_exists() → bool

Return True iff any SUT component pod exists in the cluster.

Returns:: True if at least one SUT pod exists (any state).
Return type:: bool

sut_healthy() → bool

Return True iff all SUT pods are Running and Ready.

Returns:: True if all SUT pods are running and have passed readiness checks.
Return type:: bool

sut_pending() → bool

Return True iff any SUT pod is in Pending state.

Returns:: True if at least one SUT pod is Pending.
Return type:: bool

sut_running() → bool

Return True iff all SUT pods are Running.

Returns:: True if all SUT pods are Running.
Return type:: bool

workers_healthy() → bool

Return True iff all worker pods are Running and Ready.

Returns:: True if all worker pods are running and ready.
Return type:: bool

class bexhoma.configurations.HostProbe(config: SutConfiguration)

Bases: object

Runs shell commands inside the SUT pod to collect host-level metrics.

Parameters:: config (SutConfiguration) – The parent configuration this probe belongs to.

check_dbms_connection(ip: str, port: int) → bool

Check if DBMS is open for connections by opening a socket to ip:port.

Parameters:

ip – IP of the host to connect to.
port – Port of the server on the host to connect to.

Returns:

True iff connecting is possible.

Return type:

bool

check_volumes() → None

Write volume size/used labels to PVCs by probing mounted volumes in pods.

Iterates over all tracked deployments and stateful sets, calls get_host_volume() per pod, and writes the result as labels on the associated PVC via kubectl label.

get_host_all() → dict

Call all get_host_* probes and return a consolidated dict.

Returns:: Dict of host metrics (RAM, CPU, GPU, Cores, host, node, disk, etc.).
Return type:: dict

get_host_cores() → int

Return the number of CPU cores from /proc/cpuinfo.

Returns:: Core count, or 0 on failure.
Return type:: int

get_host_cpu() → str

Return CPU model name from /proc/cpuinfo.

Returns:: CPU model name string.
Return type:: str

get_host_cpulist() → str

Return the allowed CPU list from /proc/self/status.

Returns:: CPU list string, or empty string on failure.
Return type:: str

get_host_cuda() → str

Return CUDA version string from nvidia-smi output.

Returns:: CUDA version string, stripped of surrounding whitespace and pipes.
Return type:: str

get_host_diskspace_used() → int

Return disk space used on the root filesystem in megabytes.

Returns:: Used disk space in megabytes, or 0 on failure.
Return type:: int

get_host_diskspace_used_data() → int

Return disk space used for database data directory in megabytes.

Returns:: Size in megabytes, or 0 on failure or if no datadir configured.
Return type:: int

get_host_gpu_ids() → list

Return list of GPU UUIDs from nvidia-smi -L.

Returns:: List of GPU UUID strings.
Return type:: list[str]

get_host_gpus() → str

Return GPU model summary string from nvidia-smi -L.

Returns:: Formatted string like “2 x Tesla V100-SXM2-32GB”.
Return type:: str

get_host_hugepages_free() → int

Return the HugePages_Free value from /proc/meminfo.

Returns:: Free huge pages count, or 0 on failure.
Return type:: int

get_host_hugepages_total() → int

Return the HugePages_Total value from /proc/meminfo.

Returns:: Total huge pages count, or 0 on failure.
Return type:: int

get_host_memory() → int

Return total RAM in bytes from /proc/meminfo.

Returns:: Total RAM in bytes, or 0 on failure.
Return type:: int

get_host_node() → str

Return the node name the SUT pod is scheduled on.

Returns:: Node name string, or empty string on failure.
Return type:: str

get_host_restarts(pod_sut: str = '') → str

Return the container restart counts for the SUT pod via kubectl.

Parameters:: pod_sut – Optional pod name override; defaults to current SUT pod.
Returns:: Restart count string from kubectl jsonpath output.
Return type:: str

get_host_system() → str

Return the OS kernel version via uname -r.

Returns:: Kernel version string.
Return type:: str

get_host_volume(pod: str = '') → Tuple

Return (size, used) of mounted volumes in the SUT container.

Parameters:: pod – Optional pod name override; defaults to current SUT pod.
Returns:: Tuple (size, used) as human-readable strings.
Return type:: tuple

get_time_diff() → int

Return clock skew in seconds between the SUT pod and the local host.

Runs date +"%s" inside the SUT pod and locally, then returns remote_timestamp - local_timestamp.

Returns:: Clock difference in seconds (positive means remote is ahead).
Return type:: int

class bexhoma.configurations.LifecycleManager(config: SutConfiguration)

Bases: object

Manages SUT, monitoring, maintaining, loading start/stop and port forwarding.

Parameters:: config (SutConfiguration) – The parent configuration this manager belongs to.

create_monitoring(app: str = '', component: str = 'monitoring', experiment: str = '', configuration: str = '') → str

Generate the monitoring component name.

Parameters:

app – App label.
component – Component label (default 'monitoring').
experiment – Experiment code.
configuration – DBMS configuration name.

Returns:

Generated component name string.

Return type:

str

generate_port_forward() → str

Generate a kubectl port-forward command string for this SUT.

Returns:: Ready-to-run kubectl port-forward command.
Return type:: str

get_deployment_component(container: str) → str

Find the first deployment/statefulset that contains a container with the given name.

Parameters:: container – Container name to search for.
Returns:: Component name that owns the container, or empty string if not found.
Return type:: str

start_maintaining(app: str = '', component: str = 'maintaining', experiment: str = '', configuration: str = '', parallelism: int = 1, num_pods: int = 1) → None

Start a maintaining job.

Parameters:

app – App label.
component – Component label (default 'maintaining').
experiment – Experiment code.
configuration – DBMS configuration name.
parallelism – Number of parallel pods in the job.
num_pods – Total pods that must complete.

start_monitoring(app: str = '', component: str = 'monitoring', experiment: str = '', configuration: str = '') → None

Start a Prometheus monitoring deployment for this configuration.

Parameters:

app – App label.
component – Component label (default 'monitoring').
experiment – Experiment code.
configuration – DBMS configuration name.

start_sut(app: str = '', component: str = 'sut', experiment: str = '', configuration: str = '')

Start the system-under-test (DBMS).

Controls optional worker and storage resources. Resources are set according to the configuration’s resources dict.

Parameters:

app – App label.
component – Component label (default 'sut').
experiment – Experiment code.
configuration – DBMS configuration name.

Returns:

True if the SUT was started; False if it was already running.

Return type:

bool or None

stop_loading(app: str = '', component: str = 'loading', experiment: str = '', configuration: str = '') → None

Stop a loading job and remove all its pods.

Parameters:

app – App label.
component – Component label (default 'loading').
experiment – Experiment code.
configuration – DBMS configuration name.

stop_maintaining(app: str = '', component: str = 'maintaining', experiment: str = '', configuration: str = '') → None

Stop a maintaining job and remove all its pods.

Parameters:

app – App label.
component – Component label (default 'maintaining').
experiment – Experiment code.
configuration – DBMS configuration name.

stop_monitoring(app: str = '', component: str = 'monitoring', experiment: str = '', configuration: str = '') → None

Stop a monitoring deployment and remove its service.

Parameters:

app – App label.
component – Component label (default 'monitoring').
experiment – Experiment code.
configuration – DBMS configuration name.

stop_sut(app: str = '', component: str = 'sut', experiment: str = '', configuration: str = '') → None

Stop the SUT deployment and remove services, stateful sets, and optionally storage.

Parameters:

app – App label.
component – Component label (default 'sut').
experiment – Experiment code.
configuration – DBMS configuration name.

class bexhoma.configurations.LoadingCoordinator(config: SutConfiguration)

Bases: object

Coordinates data loading into the SUT via in-cluster jobs and direct scripts.

Parameters:: config (SutConfiguration) – The parent configuration this coordinator belongs to.

check() → None

Check loading status, update timing attributes, and clean up finished jobs.

If a loading job has succeeded: reads timing labels from the SUT pod, deletes the job and its pods, triggers index loading if configured. Also reads pod labels to update loading_started / loading_finished.

copy_log() → None: Copy the DBMS log file from the SUT pod to the result folder.

get_list_of_pvc() → list

Return a flat list of all PVC names claimed by this configuration.

Returns:: List of PVC name strings.
Return type:: list[str]

get_volume_to_label() → str

Return the PVC name that should receive the storage timing labels.

Returns:: PVC name string.
Return type:: str

load_data(scripts: list, time_offset: int = 0, time_start_int: int = 0, script_type: str = 'loaded') → None

Start async loading of SQL/shell scripts into the SUT.

Calls load_data_asynch() in threads — one per tenant for schema/database tenancy, one thread otherwise.

Parameters:

scripts – List of script filenames to execute.
time_offset – Previously elapsed seconds added to the total duration.
time_start_int – Unix timestamp when loading started (0 = compute now).
script_type – Label key written to pod/PVC on completion.

prepare_init_dbms(scripts: list) → None

Copy and optionally fill DDL scripts into the SUT pod for loading.

Parameters:: scripts – List of script filenames from the experiment’s initscript list.

start_exec(delay: int = 0) → bool

Start data ingestion by running init scripts directly inside the SUT pod.

Parameters:: delay – Seconds to wait after invoking scripts.
Returns:: True if loading was initiated, False if SUT is not running.
Return type:: bool

start_pod(app: str = '', component: str = 'loading', experiment: str = '', configuration: str = '', parallelism: int = 1, num_pods: int = 1) → None

Start a Kubernetes loading job (parallel data ingestion).

Parameters:

app – App label.
component – Component label (default 'loading').
experiment – Experiment code.
configuration – DBMS configuration name.
parallelism – Number of parallel pods.
num_pods – Total pods that must complete.

class bexhoma.configurations.ManifestBuilder(config: SutConfiguration)

Bases: object

Builds and writes Kubernetes job/deployment YAML manifests.

Wraps create_manifest_job, create_manifest_benchmarking, create_manifest_maintaining, create_manifest_loading, get_patched_yaml, and patch_dbms_args.

Parameters:: config (SutConfiguration) – The parent configuration this builder belongs to.

create_manifest_benchmarking(connection: str, app: str = '', component: str = 'benchmarker', experiment: str = '', configuration: str = '', experimentRun: str = '', client: str = '1', parallelism: int = 1, alias: str = '', env: dict = {}, template: str = '', num_pods: int = 1, benchmark_run: str = '', template_override: str = '') → str

Create a benchmarker job manifest.

Template resolution priority: template_override > template argument > self.experiment.jobtemplate_benchmarking > default "jobtemplate-benchmarking-dbmsbenchmarker.yml".

Parameters:

connection – Connection/configuration name for dbmsbenchmarker.
app – App label.
component – Component label (default 'benchmarker').
experiment – Experiment code.
configuration – DBMS configuration name.
experimentRun – Repetition index string.
client – Sequential client-round index.
parallelism – Number of parallel pods.
alias – Alias name forwarded to dbmsbenchmarker.
env – Extra environment variables merged into the job ENV.
template – Optional YAML template filename override.
num_pods – Total pod count.
benchmark_run – Parallel benchmark index within a client round.
template_override – When non-empty, takes precedence over all other template resolution.

Returns:

Path to the written YAML manifest file.

Return type:

str

create_manifest_job(app: str = '', component: str = 'benchmarker', experiment: str = '', configuration: str = '', experimentRun: str = '', client: str = '1', parallelism: int = 1, env: dict = {}, template: str = '', nodegroup: str = '', num_pods: int = 1, connection: str = '', patch_yaml: str = '', benchmark_run: str = '', template_override: str = '') → str

Create a Kubernetes job manifest and write it to the experiment path.

Parameters:

app – App label for the job.
component – Component label (e.g. 'benchmarker').
experiment – Experiment code.
configuration – DBMS configuration name.
experimentRun – Repetition index string.
client – Sequential client-round index.
parallelism – Number of parallel pods.
env – Extra environment variable dict merged into the job manifest.
template – YAML template filename.
nodegroup – Node selector group key.
num_pods – Total number of pods (spec.completions).
connection – Connection name label.
patch_yaml – Optional hiyapyco YAML patch string.
benchmark_run – Parallel benchmark index within a client round.
template_override – When non-empty, overrides template.

Returns:

Path to the written YAML manifest file.

Return type:

str

create_manifest_loading(app: str = '', component: str = 'loading', experiment: str = '', configuration: str = '', parallelism: int = 1, alias: str = '', num_pods: int = 1, connection: str = '', benchmark_run: str = '', template_override: str = '') → str

Create a loading job manifest.

Parameters:

app – App label.
component – Component label (default 'loading').
experiment – Experiment code.
configuration – DBMS configuration name.
parallelism – Number of parallel pods.
alias – Alias (unused, kept for API symmetry).
num_pods – Total pods that must complete (spec.completions).
connection – Connection name label.
benchmark_run – Loader index forwarded to create_manifest_job().
template_override – When non-empty, overrides template resolution.

Returns:

Path to the written YAML manifest file.

Return type:

str

create_manifest_maintaining(app: str = '', component: str = 'maintaining', experiment: str = '', configuration: str = '', parallelism: int = 1, alias: str = '', num_pods: int = 1, connection: str = '') → str

Create a maintaining job manifest.

Parameters:

app – App label.
component – Component label (default 'maintaining').
experiment – Experiment code.
configuration – DBMS configuration name.
parallelism – Number of parallel pods.
alias – Alias forwarded to dbmsbenchmarker.
num_pods – Total pod count.
connection – Connection name label.

Returns:

Path to the written YAML manifest file.

Return type:

str

get_patched_yaml(file: str, patch: str = '') → list

Load a YAML file, optionally deep-merging a patch string, and return parsed docs.

Parameters:

file – Path to the base YAML file.
patch – Optional YAML-formatted patch string.

Returns:

List of parsed YAML document dicts.

Return type:

list

patch_dbms_args(yaml_docs: List[dict], operations: List[Tuple[dict, str]]) → List[dict]

Apply parameter-patch operations across all documents in a manifest.

Parameters:

yaml_docs – List of parsed YAML document dicts.
operations – List of (selector_dict, value_str) pairs.

Returns:

Modified list of YAML document dicts.

Return type:

list[dict]

class bexhoma.configurations.MetricsCollector(config: SutConfiguration)

Bases: object

Builds connection configs and fetches Prometheus metrics for a configuration.

Parameters:: config (SutConfiguration) – The parent configuration this collector belongs to.

fetch(connection: str, connection_file: str, container: str, component: str, component_type: str, title: str, experiment: str, time_start: int, time_end: int, metrics_type: str, pod_dashboard: str, optional: bool = False) → None

Fetch Prometheus metrics into the experiment result via the dashboard pod.

Parameters:

connection – dbmsbenchmarker connection name.
connection_file – Filename of the connection config (e.g. "PostgreSQL.config").
container – Container name whose metrics to query (e.g. "dbms").
component – Component label used in metric routing.
component_type – Key under monitoring_components in the workload dict.
title – Human-readable label stored in monitoring_components.
experiment – Experiment code.
time_start – Unix timestamp of metric window start.
time_end – Unix timestamp of metric window end.
metrics_type – Key selecting the metric set in the connection config.
pod_dashboard – Name of the dashboard pod to run metrics.py in.
optional – When True, the component is added to optional_monitoring_components.

get_connection_config(connection: str, alias: str = '', dialect: str = '', serverip: str = 'localhost', monitoring_host: str = 'localhost') → dict

Build the dbmsbenchmarker connection config dict for this SUT.

Collects host system information, worker information, resource limits, connection management settings, and monitoring metric queries.

Parameters:

connection – Name of the dbmsbenchmarker connection.
alias – Optional display alias for the connection.
dialect – Optional SQL dialect string.
serverip – Service IP/hostname used in JDBC URL formatting.
monitoring_host – Prometheus monitoring service name.

Returns:

Connection config dict compatible with dbmsbenchmarker.

Return type:

dict

class bexhoma.configurations.SutConfiguration(experiment, docker: str | None = None, configuration: str = '', script: str | None = None, alias: str | None = None, num_experiment_to_apply: int | None = None, clients: list = None, dialect: str = '', worker: int = 0, dockerimage: str = '')

Bases: object

Manage a single DBMS configuration plugged into a bexhoma experiment.

A configuration object is created for each DBMS variant under test and attached to a base experiment. It holds all deployment parameters, tracks loading and benchmarking state, and provides access to six specialised helper objects:

status — ComponentStatus
host — HostProbe
lifecycle — LifecycleManager
loader — LoadingCoordinator
runner — BenchmarkRunner
metrics — MetricsCollector
manifest — ManifestBuilder

add_benchmark_list(list_clients: list) → None

Add a list of benchmarker-instance counts for the current SUT.

Example: [1, 2, 1] schedules three sequential rounds with 1, 2, and 1 benchmarker instances. Also reconstructs experiment_dict["benchmarker"] so that each client round carries the correct parallelism and the per-round parameters accumulated in benchmarking_parameters_list.

Parameters:: list_clients (list) – List of benchmarker-instance counts.

add_benchmarking_parameters(parallelism: int = None, **env_vars) → None

Add a new sequential client round to the experiment dict.

Clones the first benchmarker entry’s header keys and merges env_vars on top of that entry’s parameters. When parallelism is None, inherits the template entry’s parallelism.

Also appends env_vars to benchmarking_parameters_list for backward compatibility.

Parameters:

parallelism (int) – Pod count for this client round; inherits template if None.
env_vars – ENV vars injected into the job container for this round.

add_loading_parameters(name: str, template: str, benchmarker: str, parallelism: int = 1, num_pods: int = None, target: str = 'sut', **env_vars) → None

Add a parallel loader entry to the experiment dict.

All loader entries run simultaneously during the loading phase.

Parameters:

name – Short identifier for this loader entry.
template – K8s job template filename.
benchmarker – Tool name — identifies the evaluator’s log_to_df_loading().
parallelism – Max concurrent pods (K8s spec.parallelism).
num_pods – Total pods that must complete (K8s spec.completions). Defaults to parallelism.
target – Component the job runs against (default 'sut').
env_vars – ENV vars injected into the container.

add_parallel_benchmark(name: str, template: str, benchmarker: str, parallelism: int = 1, target: str = 'sut', **env_vars) → None

Add a parallel benchmark to the last client round.

Creates a new entry inside the last inner list of experiment_dict["benchmarker"], enabling multiple benchmarks to run simultaneously within one client round.

Parameters:

name – Short identifier for this benchmark entry.
template – K8s job template filename.
benchmarker – Tool name ('ycsb', 'hammerdb', 'benchbase', 'dbmsbenchmarker').
parallelism – Number of pods.
target – Component the job runs against (default 'sut').
env_vars – ENV vars injected into the container.

additional_labels: Extra Kubernetes labels added to all managed pods.

alias: Human-readable alias for this configuration in result reports.

appname: Kubernetes app label used to identify bexhoma resources.

are_worker_ready: True once all worker pods report ready.

attach_worker() → None: Attach worker nodes to the SUT master via the attachWorker command.

benchmark: Optional dbmsbenchmarker instance.

benchmark_list: Ordered list of benchmarker-instance counts (consumed as a queue).

benchmark_list_template: Original copy of benchmark_list kept as a template.

benchmarking_parameters: Dict of parameters forwarded to the benchmarking tool.

benchmarking_parameters_list: List of per-run benchmarking parameter dicts.

benchmarking_parameters_list_template: Original copy of benchmarking_parameters_list.

benchmarking_patch: Patch dict applied to the benchmarking job YAML manifest.

benchmarking_timespans: Per-container (start, end) pairs for benchmarking pods.

check_sut() → bool

Check if the SUT pod is running and store its name in pod_sut.

Returns:: True if at least one SUT pod was found.
Return type:: bool

client: Current position in the benchmarker sequence.

code: Unique experiment run code shared across all configurations in this run.

configuration: Name of the configuration; default: docker image name with counter suffix.

configurations: dict = {}: Class-level counter used to give each configuration a unique suffix.

connection: Name of the dbmsbenchmarker connection currently being executed.

connection_management: Dict of connection management parameters.

connection_parameter: Collects parameters that may be interesting for result evaluation.

current_benchmark_connection: Name of the connection currently being benchmarked.

current_benchmark_start: Unix timestamp when the current benchmark run started.

ddl_parameters: DDL schema parameters for init scripts.

deployment_infos: Info about deployed deployments, stateful sets, PVCs, pods, and containers.

dialect: SQL dialect string forwarded to dbmsbenchmarker.

docker: Name of the Docker image.

dockerimage: Docker image name of the SUT.

download_experiment_file(filename: str)

Download a file from the experiment’s result storage.

Parameters:: filename – Path of the file to download.
Returns:: Result of the download operation.

eval_parameters: Parameters forwarded to dbmsbenchmarker for evaluation.

execute_command_in_pod_sut(command: str, pod: str = '', container: str = '', params: str = '')

Run a shell command inside the SUT container.

Defaults to the current SUT pod and the container named "dbms".

Parameters:

command – Shell command to run.
pod – Pod name; defaults to pod_sut.
container – Container name; defaults to sut_container_name.
params – Optional additional parameters (currently unused).

Returns:

stdout of the shell command.

experiment: Parent experiment object.

experiment_dict: dict: Central experiment dict describing all loader and benchmarker jobs.

experiment_done: True once the SUT has performed the experiment completely.

experiment_name: Identifier of experiment; may be overwritten when stateful set PVCs forbid per-experiment names.

generate_component_name(app: str = '', component: str = '', experiment: str = '', configuration: str = '', experiment_run: str = '', client: str = '', benchmark_run: str = '') → str

Generate a Kubernetes-compatible name for a component.

Format: {app}-{component}-{configuration}-{experiment}[-{experiment_run}][-{client}[-{benchmark_run}]]

Parameters:

app – App the component belongs to.
component – Component type, e.g. 'sut' or 'benchmarker'.
experiment – Unique experiment identifier.
configuration – DBMS configuration name.
experiment_run – Repetition index (omitted when empty).
client – Sequential client-round index (omitted when empty).
benchmark_run – Parallel benchmark index within a client round (omitted when empty).

Returns:

Lower-case component name string.

Return type:

str

get_experiment_name() → str

Return the experiment run code identifying this experiment across all configurations.

Returns:: Experiment code string.
Return type:: str

get_list_of_pvc() → list

Return a flat list of all PVC names currently claimed by this configuration.

Returns:: List of PVC name strings.
Return type:: list[str]

get_service_sut(configuration: str) → str

Return the name of the Kubernetes service for the SUT.

Parameters:: configuration – Name of the DBMS configuration.
Returns:: Service name.
Return type:: str

get_worker_endpoints() → list

Return endpoints of the headless service monitoring distributed DBMS workers.

When statefulset_name is set (external stateful set, e.g. YugabyteDB), returns bare pod names. Otherwise returns {pod}.{service} entries.

Returns:: List of endpoint strings.
Return type:: list[str]

get_worker_name(component: str = 'worker') → str

Return a name template for worker pods.

When worker_name_app or worker_name_component are set, the override values are used together with experiment_name (the experiment code) instead of storage_label. This covers distributed DBMS workers (e.g. Dragonfly, Redis) that use shorter pod names tied to the experiment code rather than the storage label.

Parameters:: component – Component type to use when not overridden.
Returns:: Worker name template string.
Return type:: str

get_worker_pods(component: str = 'worker', only_stateful: bool = False) → list

Return a list of all worker pod names for the current SUT.

When statefulset_name is set, delegates to the cluster’s stateful set pod discovery (e.g. for YugabyteDB tservers). Otherwise finds pods via the standard bexhoma label selector.

Parameters:

component – Component label used in the label selector.
only_stateful – When True, return only stateful set pods (pods whose names end with a numeric index).

Returns:

List of pod name strings.

Return type:

list[str]

indexing: Key of the indexing script, or falsy if no separate indexing step.

indexscript: Empty when no separate indexing step is configured.

initscript: Init script definition dict.

is_sut_ready: True once the SUT pod reports ready.

property jobtemplate_loading: str

Name of the YAML template file used for the loading job.

Returns:: Template file name.
Return type:: str

jobtemplate_maintaining: Name of YAML template file for the maintaining job.

loading_active: True iff a loading component should be deployed.

loading_after_time: Optional unix timestamp after which loading should start.

loading_deactivated: Do not load at all and do not test for loading.

loading_finished: True once loading has completed.

loading_parameters: Parameters for the loading component.

loading_patch: Patch dict applied to the loading job YAML manifest.

loading_started: True once loading has been initiated.

loading_timespans: Per-container (start, end) pairs for loading pods.

logger: Logger for this configuration.

maintaining_active: True iff a maintaining component should be deployed after loading.

maintaining_parameters: Parameters for the maintaining component.

max_sut_dbms: Max SUT pods of this DBMS type (config.docker) allowed in the cluster at once; None means no limit.

monitor_app_active: True iff application-level monitoring is active.

monitor_loading: Fetch metrics for the loading phase when monitoring is active.

monitoring_active: True iff Prometheus-based cluster monitoring is active.

monitoring_sut: Fetch SUT metrics when monitoring is active.

nodes: Dict of node infos to guide component placement.

num_experiment_to_apply: Number of benchmarking runs to perform.

num_experiment_to_apply_done: Number of benchmarking runs completed so far.

num_loading: Number of parallel loading threads.

num_loading_pods: Number of loading pods currently active.

num_maintaining: Number of parallel maintaining threads.

num_maintaining_pods: Number of maintaining pods currently active.

num_tenants: Number of tenants for multi-tenant experiments.

num_worker: Number of worker pods to deploy alongside the SUT.

patch_benchmarking(patch) → None

Apply a YAML patch string to the benchmarking job manifest.

Parameters:: patch – YAML-formatted patch string.

patch_loading(patch) → None

Apply a YAML patch string to the loading job manifest.

Parameters:: patch – YAML-formatted patch string.

path: Filesystem path to the experiment’s working directory.

path_experiment_docker: Experiment subfolder matching the docker image name.

pod_sut: Name of the SUT’s master pod.

prometheus_interval: Prometheus scrape interval in seconds.

prometheus_timeout: Prometheus scrape timeout in seconds.

reset_sut() → None: Forget that the SUT has been loaded and benchmarked.

resources: Dict of Kubernetes resource requests/limits for the SUT pod.

script: Key of the init script used for loading.

service: Name of the Kubernetes Service currently exposing the SUT.

set_additional_labels(**kwargs) → None

Set extra Kubernetes labels added to all managed pods.

Parameters:: kwargs – Label dict, e.g. SF=100.

set_benchmarking_parameters(**kwargs) → None

Set ENV vars for the benchmarking component.

Merges experiment-wide defaults first; per-configuration kwargs win on conflict. Also updates all entries in the first benchmarker round in experiment_dict.

Parameters:: kwargs – Parameter dict.

set_connection_management(**kwargs) → None

Set connection management data for the benchmarker component.

Parameters:: kwargs – Dict of connection management parameters, e.g. timeout=60.

set_ddl_parameters(**kwargs) → None

Set DDL parameters that are substituted in init scripts.

Parameters:: kwargs – Parameter dict, e.g. index='btree'.

set_eval_parameters(**kwargs) → None

Set evaluation parameters forwarded to the benchmarker component.

Parameters:: kwargs – Parameter dict, e.g. type='noindex'.

set_experiment(instance=None, volume=None, docker=None, script=None, indexing=None) → None

Read experiment details from the cluster config.

Parameters:

instance – Unused.
volume – Override the persistent volume.
docker – Unused.
script – Override the init script key.
indexing – Override the indexing script key.

set_experiment_dict(d: dict) → None

Replace the entire experiment dict.

Parameters:: d (dict) – New experiment dict with "loader" and "benchmarker" keys.

set_loading(parallel: int, num_pods: int = None) → None

Set job parameters for loading: parallel pods and total pod count.

Parameters:

parallel – Number of parallel pods.
num_pods – Total number of pods; defaults to parallel.

set_loading_parameters(**kwargs) → None

Set ENV vars for the loading component.

Merges experiment-wide defaults first; per-configuration kwargs win on conflict. Also updates the first loader entry in experiment_dict when present.

Parameters:: kwargs – Parameter dict, e.g. PARALLEL='64'.

set_maintaining(parallel: int, num_pods: int = None) → None

Set job parameters for the maintaining component.

Parameters:

parallel – Number of parallel pods.
num_pods – Total number of pods; defaults to parallel.

set_maintaining_parameters(**kwargs) → None

Set ENV vars for the maintaining component.

Parameters:: kwargs – Parameter dict, e.g. PARALLEL='64'.

set_metric_of_config(metric: str, host: str, gpuid: str, schema: str, database: str, component: str = '') → str

Substitute placeholders in a PromQL query, routing by deployment type.

When worker_metric_strip_container is True (e.g. YugabyteDB), strips the container="dbms" filter from the query and uses the component name directly as the configuration label — matching the external stateful set naming convention.

Parameters:

metric – Parametrised PromQL query string.
host – Hostname of the node to monitor.
gpuid – GPU identifier (or empty string).
schema – Database schema name.
database – Database name.
component – Stateful set component name; optional.

Returns:

Filled PromQL query string.

Return type:

str

set_metric_of_config_default(metric: str, host: str, gpuid: str, schema: str, database: str, experiment: str = None) → str

Substitute placeholders in a PromQL query for bexhoma-managed components.

Parameters:

metric – Parametrised PromQL query string.
host – Hostname of the node to monitor.
gpuid – GPU identifier (or empty string).
schema – Database schema name.
database – Database name.
experiment – Experiment code; defaults to code.

Returns:

Filled PromQL query string.

Return type:

str

set_nodes(**kwargs) → None

Set node selector parameters for experiment components.

Parameters:: kwargs – Node info dict, e.g. sut='sut', loading='auxiliary'.

set_resources(**kwargs) → None

Set Kubernetes resource requests/limits for the SUT.

Parameters:: kwargs – Resource dict, e.g. requests={'cpu': 4}.

set_storage(**kwargs) → None

Set storage parameters for the SUT’s persistent volume.

Parameters:: kwargs – Storage dict, e.g. storageSize='100Gi'.

set_sut_parameters(**kwargs) → None

Set ENV vars for the SUT and worker components.

Parameters:: kwargs – Parameter dict, e.g. PARALLEL='64'.

statefulset_name: Name of the stateful set managing pods of a distributed DBMS.

storage: Parameters for persistent storage.

storage_label: Kubernetes node label used to select the storage node for PVs.

sut_container_name: Container name in the SUT pod used for monitoring and exec.

sut_has_pool: True iff there is a pool component (for monitoring).

sut_parameters: Parameters for the SUT and worker components.

sut_pod_name: Fixed pod name for SUTs not controlled by bexhoma.

sut_service_name: Fixed service name for SUTs not controlled by bexhoma.

sut_startup_args: Args set for the SUT container in YAML at startup.

sut_template: Name of YAML manifest in k8s/ for SUT deployment.

tenant_per: Tenancy mode: ‘’, ‘schema’, ‘database’, or ‘container’.

tenant_ready_to_index: True once this tenant’s SUT is ready to accept indexing.

tenant_ready_to_load: True once this tenant’s SUT is ready to accept loading.

tenant_started_to_index: True once indexing for this tenant has been initiated.

tenant_started_to_load: True once loading for this tenant has been initiated.

time_generating: Seconds taken for data generation.

time_index: Seconds taken for index creation.

time_ingesting: Seconds taken for ingesting existing data.

time_loading: Seconds taken for the initial data load.

time_loading_end: Unix timestamp when loading ended.

time_loading_start: Unix timestamp when loading started.

time_schema: Seconds taken for schema creation.

times_scripts: Per-script timing dict.

upload_experiment_file(filename: str)

Upload a file to the experiment’s result storage.

Parameters:: filename – Path of the file to upload.
Returns:: Result of the upload operation.

use_distributed_datasource: True iff the SUT should mount ‘benchmark-data-volume’ as source of non-generated data.

use_ephemeral_storage() → bool

Return True iff ephemeral-storage resources should be requested for the SUT.

This is the case when a storage size is configured but no storage class is set, meaning no persistent volume is created and the database files live in the container’s ephemeral storage.

Return type:: bool

use_ramdisk() → bool

Return True iff the storage class is 'ramdisk'.

Return type:: bool

use_storage() → bool

Return True iff persistent storage is configured for the SUT.

Return type:: bool

volume: Name of the persistent volume used by this configuration.

volume_per_tenant: True iff each tenant gets its own persistent volume.

volumeid: Identifier of the persistent volume claimed by this configuration.

wait(sec: int, silent: bool = False) → None

Wait for a number of seconds, optionally printing a message.

Parameters:

sec – Number of seconds to wait.
silent – When True, suppress output.

worker_metric_strip_container: When True, strip container filter from Prometheus metric queries (e.g. for YugabyteDB).

worker_name_app: App label override for get_worker_name(); empty means use self.appname.

worker_name_component: Component label override for get_worker_name(); empty means use the method argument.

worker_startup_args: Args set for the worker container in YAML at startup.

bexhoma.configurations.default: Backward-compatibility alias — new code should use SutConfiguration.

bexhoma.configurations.ensure_arg_pairs(args_list: List[str] | None, updates: List[Tuple[str, str]]) → List[str]

Update or append -c key=value pairs in a container args list.

Parameters:

args_list – Existing container args (e.g. ["-c","max_connections=3000"]).
updates – List of (key, value) pairs to apply.

Returns:

New args list with updates applied.

Return type:

list[str]

bexhoma.configurations.find_workloads(doc: dict, kind: str, name: str) → bool

Return True if this YAML document matches the requested kind and name.

Parameters:

doc – Parsed YAML document dict.
kind – Resource kind ('deployment' or 'statefulset').
name – Expected metadata.name value.

Returns:

True iff the document kind and name match.

Return type:

bool

bexhoma.configurations.load_data_asynch(app, component, experiment, configuration, pod_sut, scriptfolder, commands, loadData, path, volume, context, service_name, time_offset=0, time_start_int=0, script_type='loaded', namespace='', num_tenants=0, id_tenant=0, database=[])

Execute loading scripts inside the SUT pod in a background thread.

Runs SQL and shell scripts in sequence inside the pod, labels the SUT pod and its PVC with timing and completion metadata, and handles multi-tenant ordering via a count-down label on the pod.

Parameters:

app – App label for the SUT pod.
component – Component label ('sut').
experiment – Experiment code.
configuration – DBMS configuration name.
pod_sut – Name of the SUT pod to execute commands in.
scriptfolder – Remote path inside the pod where scripts reside.
commands – List of script filenames to execute.
loadData – Shell command template with {scriptname} placeholder.
path – Local result folder path for log files.
volume – PVC name to label (empty string skips PVC labelling).
context – Kubernetes context string for kubectl.
service_name – Service name injected into loadData as {service_name}.
time_offset – Previously elapsed seconds added to the total.
time_start_int – Unix timestamp of when loading started (0 = compute now).
script_type – Label key written to pod/PVC (e.g. 'loaded', 'indexed').
namespace – Kubernetes namespace injected into loadData.
num_tenants – Total number of tenants; 0 means single-tenant mode.
id_tenant – Index of this tenant (0-based).
database – List of database names to iterate scripts over.

bexhoma.configurations.patch_container(doc: dict, container_name: str, param: str, value: str) → bool

Patch a single container’s args in a Deployment/StatefulSet YAML doc.

Parameters:

doc – Parsed YAML document dict.
container_name – Name of the container to patch.
param – Parameter key to set inside -c key=value.
value – Parameter value.

Returns:

True iff any changes were made.

Return type:

bool