bexhoma.collectors.benchbase module

Collector for Benchbase experiments.

Provides BenchbaseCollector, a thin subclass of CollectorBase that wires up evaluators.benchbase as the evaluator. All data collection and aggregation logic is inherited from CollectorBase.

Authors: Patrick K. Erdelt Copyright (C) 2020 Patrick K. Erdelt SPDX-License-Identifier: AGPL-3.0-or-later See LICENSE for details.

class bexhoma.collectors.benchbase.BenchbaseCollector(path, codes, benchmark_run: int = 0)

Bases: CollectorBase

Collector for Benchbase experiments.

Overrides get_evaluator() to return a evaluators.benchbase instance. All data collection and aggregation methods are inherited from CollectorBase.

get_benchmark_timeseries_all(metric='throughput')

Collects long-format Benchbase time-series data for a given metric across all experiment codes.

For each code and each unique (configuration, client, experiment_run) combination, calls evaluators.benchbase.get_benchmark_logs_timeseries_df_aggregated(), reshapes the result to long format, and annotates each row with its identifying fields. Connection metadata (e.g. type_tenants, num_tenants, vol_tenants) is joined in from get_connections() for each code.

Parameters:

metric (str) – Benchbase metric to retrieve (default 'throughput').

Returns:

Long-format DataFrame with columns second, code, configuration, client, experiment_run, metric, value, plus connection metadata columns.

Return type:

pandas.DataFrame

get_benchmark_timeseries_per_phase(metric='throughput')

Combines aggregated Benchbase time-series per phase from all experiment codes into a wide-format DataFrame.

For each code and each unique (configuration, client, experiment_run, benchmark_run) combination, calls evaluators.benchbase.get_benchmark_logs_timeseries_df_aggregated() and places the metric column as one column in the result. Each column is labelled {code}-{configuration}-{client}-{experiment_run}-{benchmark_run}.

Parameters:

metric (str) – Benchbase metric to retrieve (default 'throughput').

Returns:

Wide-format DataFrame indexed by second with one column per phase, or an empty DataFrame when no data is available.

Return type:

pandas.DataFrame

get_evaluator(code='')

Returns a evaluators.benchbase instance for the given experiment code.

Parameters:

code (str) – Experiment identifier. Defaults to the first code in self.codes.

Returns:

Benchbase evaluator for the specified experiment.

Return type:

evaluators.benchbase