Benchmark: Hardware
Hardware is not a DBMS benchmark — it runs fio (disk I/O) or
sysbench (CPU/memory) directly against a dedicated SUT
container over SSH, bypassing any database engine entirely. There is no data loading phase and
no -dbms engine choice beyond the single Hardware target (see DBMS.md).
The purpose of these benchmarks is not to rank hardware, but to calibrate DBMS configuration
against the actual storage a cluster provides — for example finding the queue depth
PostgreSQL’s effective_io_concurrency should target, a realistic
random_page_cost, or the raw fsync latency that bounds commit throughput under
synchronous_commit=on. This page walks through the fio sweeps in
scripts/test-docs-hardware.ps1 /
scripts/test-docs-hardware.sh
and explains what each one is for.
The results are not official benchmark results. Exact performance depends on a number of parameters, including the underlying storage class, node hardware, and cluster load at the time of the run. These examples are solely to illustrate how to use bexhoma and show the result evaluation.
Result tables below are real output from an actual cluster run of every command on this page.
References:
fio documentation: https://fio.readthedocs.io/en/latest/fio_doc.html
fio
--fsync/--fdatasync: https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-fsyncPostgreSQL WAL configuration: https://www.postgresql.org/docs/current/wal-configuration.html
PostgreSQL
effective_io_concurrency: https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-EFFECTIVE-IO-CONCURRENCY
Perform Benchmark
You will have to change the node selectors to names of nodes that exist in your cluster (or leave the corresponding parameters out):
BEXHOMA_NODE_SUT="cl-worker36"
BEXHOMA_NODE_BENCHMARK="cl-worker19"
LOG_DIR="./logs_tests"
BEXHOMA_MS=1
BEXHOMA_STORAGE_CLASS="shared"
mkdir -p $LOG_DIR
Unlike every other entry script, hardware.py has no loader — there is nothing to import
before benchmarking, so every command below goes straight to run. The page covers two groups
of commands: twelve -xht fio disk-I/O commands, and four -xht sysbench CPU/memory commands.
The twelve fio commands all share the same Hardware-1 SUT/PVC and run sequentially; two
hardware.py run invocations must never overlap in time, because the PVC name is fixed
(storage_label='hardware', not scoped by experiment code) and a second SUT pod would either
fail to attach the volume or silently write into the same test-file path as the first (see the
project notes on -rsr and PVC sharing). Each of them passes -rsr so it starts from a freshly
recreated, empty volume rather than inheriting whatever an earlier command left on the PVC — fio
creates one full -xts-sized file per numjobs thread with no cleanup between rounds by
default, so -rss is sized per command to the largest single round it will run (most stay at the
default 50Gi; the numjobs and group-commit sweeps go up to 80Gi/150Gi since they sweep
numjobs as high as 16/32).
The four sysbench commands (13-16) are CPU/memory only — no disk I/O, so none of them use
-rst/-rss/-rsr and the PVC-sharing constraint above doesn’t apply to them.
The fio workload flags (-xfrw, -xfbs, -xfid, -xfe, -xfsy, -xffd, -xfmx) each accept
a comma-separated list. Every combination across the lists is run as one more sequential round
against the same SUT, so a parameter sweep is expressed as a single invocation instead of one
process per value — see python hardware.py -h at the bottom of this page.
Fio disk benchmarks
The twelve commands below (-xht fio) cover raw disk I/O: queue-depth and block-size
characterization, PostgreSQL-page-size calibration, and WAL/durability/checkpoint simulation.
1. Queue-depth sweep
For performing the experiment we can run the hardware file.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randread,randwrite \
-xfbs 4k \
-xfid 1,2,4,8,16,32,64,128 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_depth_sweep.log
This
loops over 8 queue depths (
-xfid, doubling from 1 to 128) for bothrandreadandrandwrite(-xfrw) at a fixed 4k block size (-xfbs)runs each of the 16 combinations as one more sequential round against the same SUT (16 rounds × 60s ≈ 16 minutes)
collects SUT resource metrics (
-m)tests if results match workflow (
-tr)shows a summary
Doubling the queue depth (rather than a linear step) is standard practice for this kind of sweep: IOPS/latency behavior versus depth is fundamentally logarithmic, so evenly-spaced points on a log scale reveal the curvature efficiently, and real systems mostly configure queue depth in powers of 2 anyway (NVMe queues, io_uring, RAID controllers).
Result
docs_hardware_fio_depth_sweep.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 2294s
* Code: 1783188404
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randread', 'randwrite'].
* Block size(s) swept: ['4k'].
* Queue depth(s) swept: [1, 2, 4, 8, 16, 32, 64, 128].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819565
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-10-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824021
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-11-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:817879
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-12-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818006
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-13-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819794
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-14-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818889
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-15-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821465
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-16-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826315
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822628
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822032
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821880
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823130
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:817718
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-7-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822412
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-8-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:817960
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
* Hardware-1-1-9-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818215
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783188404
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783188404-77b66d7746-mcz5n: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 15: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 16: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 15: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 16: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:--------------------|:----------------|:------------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 72 | randread | 4k | 1 | libaio | 0 | 0 | 50 | 1 | 67.09 | 0.00 | 36.44 | 0.00 | 86.51 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 70 | randread | 4k | 2 | libaio | 0 | 0 | 50 | 1 | 153.10 | 0.00 | 31.85 | 0.00 | 76.02 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 71 | randread | 4k | 4 | libaio | 0 | 0 | 50 | 1 | 303.88 | 0.00 | 36.44 | 0.00 | 98.04 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 71 | randread | 4k | 8 | libaio | 0 | 0 | 50 | 1 | 739.75 | 0.00 | 32.64 | 0.00 | 81.26 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 70 | randread | 4k | 16 | libaio | 0 | 0 | 50 | 1 | 1676.72 | 0.00 | 26.08 | 0.00 | 72.88 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 73 | randread | 4k | 32 | libaio | 0 | 0 | 50 | 1 | 2344.08 | 0.00 | 39.58 | 0.00 | 126.35 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7-1-1 | Hardware-1-1-7 | Hardware-1-1-7-1 | 1 | 7 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 4036.87 | 0.00 | 52.17 | 0.00 | 166.72 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8-1-1 | Hardware-1-1-8 | Hardware-1-1-8-1 | 1 | 8 | 1 | 1 | 71 | randread | 4k | 128 | libaio | 0 | 0 | 50 | 1 | 5875.08 | 0.00 | 81.26 | 0.00 | 337.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9-1-1 | Hardware-1-1-9 | Hardware-1-1-9-1 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 1 | libaio | 0 | 0 | 50 | 1 | 0.00 | 82.79 | 0.00 | 41.68 | 0.00 | 94.90 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10-1-1 | Hardware-1-1-10 | Hardware-1-1-10-1 | 1 | 10 | 1 | 1 | 63 | randwrite | 4k | 2 | libaio | 0 | 0 | 50 | 1 | 0.00 | 138.53 | 0.00 | 54.79 | 0.00 | 102.24 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11-1-1 | Hardware-1-1-11 | Hardware-1-1-11-1 | 1 | 11 | 1 | 1 | 63 | randwrite | 4k | 4 | libaio | 0 | 0 | 50 | 1 | 0.00 | 263.75 | 0.00 | 58.98 | 0.00 | 122.16 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12-1-1 | Hardware-1-1-12 | Hardware-1-1-12-1 | 1 | 12 | 1 | 1 | 63 | randwrite | 4k | 8 | libaio | 0 | 0 | 50 | 1 | 0.00 | 385.27 | 0.00 | 73.92 | 0.00 | 189.79 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13-1-1 | Hardware-1-1-13 | Hardware-1-1-13-1 | 1 | 13 | 1 | 1 | 63 | randwrite | 4k | 16 | libaio | 0 | 0 | 50 | 1 | 0.00 | 768.98 | 0.00 | 76.02 | 0.00 | 189.79 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14-1-1 | Hardware-1-1-14 | Hardware-1-1-14-1 | 1 | 14 | 1 | 1 | 62 | randwrite | 4k | 32 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1187.86 | 0.00 | 108.53 | 0.00 | 219.15 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-15-1-1 | Hardware-1-1-15 | Hardware-1-1-15-1 | 1 | 15 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1783.67 | 0.00 | 143.65 | 0.00 | 261.10 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-16-1-1 | Hardware-1-1-16 | Hardware-1-1-16-1 | 1 | 16 | 1 | 1 | 63 | randwrite | 4k | 128 | libaio | 0 | 0 | 50 | 1 | 0.00 | 3287.80 | 0.00 | 162.53 | 0.00 | 261.10 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:----------------|:----------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 72 | randread | 4k | 1 | libaio | 0 | 0 | 50 | 67.09 | 0.00 | 36.44 | 0.00 | 86.51 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 70 | randread | 4k | 2 | libaio | 0 | 0 | 50 | 153.10 | 0.00 | 31.85 | 0.00 | 76.02 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 71 | randread | 4k | 4 | libaio | 0 | 0 | 50 | 303.88 | 0.00 | 36.44 | 0.00 | 98.04 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 71 | randread | 4k | 8 | libaio | 0 | 0 | 50 | 739.75 | 0.00 | 32.64 | 0.00 | 81.26 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 70 | randread | 4k | 16 | libaio | 0 | 0 | 50 | 1676.72 | 0.00 | 26.08 | 0.00 | 72.88 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 73 | randread | 4k | 32 | libaio | 0 | 0 | 50 | 2344.08 | 0.00 | 39.58 | 0.00 | 126.35 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7 | Hardware-1-1-7 | 1 | 7 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 4036.87 | 0.00 | 52.17 | 0.00 | 166.72 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8 | Hardware-1-1-8 | 1 | 8 | 1 | 1 | 71 | randread | 4k | 128 | libaio | 0 | 0 | 50 | 5875.08 | 0.00 | 81.26 | 0.00 | 337.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9 | Hardware-1-1-9 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 1 | libaio | 0 | 0 | 50 | 0.00 | 82.79 | 0.00 | 41.68 | 0.00 | 94.90 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10 | Hardware-1-1-10 | 1 | 10 | 1 | 1 | 63 | randwrite | 4k | 2 | libaio | 0 | 0 | 50 | 0.00 | 138.53 | 0.00 | 54.79 | 0.00 | 102.24 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11 | Hardware-1-1-11 | 1 | 11 | 1 | 1 | 63 | randwrite | 4k | 4 | libaio | 0 | 0 | 50 | 0.00 | 263.75 | 0.00 | 58.98 | 0.00 | 122.16 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12 | Hardware-1-1-12 | 1 | 12 | 1 | 1 | 63 | randwrite | 4k | 8 | libaio | 0 | 0 | 50 | 0.00 | 385.27 | 0.00 | 73.92 | 0.00 | 189.79 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13 | Hardware-1-1-13 | 1 | 13 | 1 | 1 | 63 | randwrite | 4k | 16 | libaio | 0 | 0 | 50 | 0.00 | 768.98 | 0.00 | 76.02 | 0.00 | 189.79 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14 | Hardware-1-1-14 | 1 | 14 | 1 | 1 | 62 | randwrite | 4k | 32 | libaio | 0 | 0 | 50 | 0.00 | 1187.86 | 0.00 | 108.53 | 0.00 | 219.15 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-15 | Hardware-1-1-15 | 1 | 15 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1783.67 | 0.00 | 143.65 | 0.00 | 261.10 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-16 | Hardware-1-1-16 | 1 | 16 | 1 | 1 | 63 | randwrite | 4k | 128 | libaio | 0 | 0 | 50 | 0.00 | 3287.80 | 0.00 | 162.53 | 0.00 | 261.10 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 58.46 | 2.03 | 0.02 | 4.02 |
| Hardware-1-1-2-1 | 59.58 | 2.00 | 0.02 | 0.02 |
| Hardware-1-1-3-1 | 61.58 | 1.67 | 0.02 | 0.02 |
| Hardware-1-1-4-1 | 62.67 | 2.12 | 0.03 | 4.03 |
| Hardware-1-1-5-1 | 62.89 | 2.34 | 0.02 | 0.02 |
| Hardware-1-1-6-1 | 64.63 | 1.60 | 0.03 | 4.03 |
| Hardware-1-1-7-1 | 52.98 | 1.43 | 0.02 | 0.02 |
| Hardware-1-1-8-1 | 52.85 | 1.30 | 0.02 | 0.02 |
| Hardware-1-1-9-1 | 58.73 | 1.62 | 0.02 | 0.02 |
| Hardware-1-1-10-1 | 62.15 | 2.77 | 0.02 | 0.02 |
| Hardware-1-1-11-1 | 62.97 | 2.18 | 0.02 | 0.02 |
| Hardware-1-1-12-1 | 62.34 | 1.46 | 0.03 | 0.03 |
| Hardware-1-1-13-1 | 63.76 | 2.10 | 0.02 | 0.02 |
| Hardware-1-1-14-1 | 65.04 | 1.83 | 0.02 | 0.02 |
| Hardware-1-1-15-1 | 67.95 | 2.45 | 0.02 | 0.02 |
| Hardware-1-1-16-1 | 71.27 | 2.43 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.52 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.54 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.52 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.54 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.49 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.44 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-7-1 | 0.51 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-8-1 | 0.49 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-9-1 | 0.51 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-10-1 | 0.49 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-11-1 | 0.51 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-12-1 | 0.45 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-13-1 | 0.51 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-14-1 | 0.44 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-15-1 | 0.46 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-16-1 | 0.52 | 0.02 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per queue depth, with IOPS and completion-latency percentiles aggregated across all pods in that round.
2. Depth-sweep refinement around the elbow
The coarse sweep above only localizes the elbow to “somewhere between 64 and 128” — each doubling step covers a wide range. This does a linear pass inside that bracket to pinpoint the actual knee instead of just the bracket containing it.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randread,randwrite \
-xfbs 4k \
-xfid 64,80,96,112,128 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_depth_sweep_refine.log
10 rounds (2 patterns × 5 depths) ≈ 10 minutes.
Result
docs_hardware_fio_depth_sweep_refine.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 1436s
* Code: 1783194420
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randread', 'randwrite'].
* Block size(s) swept: ['4k'].
* Queue depth(s) swept: [64, 80, 96, 112, 128].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822252
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-10-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:830607
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825422
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:830646
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826119
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821292
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819472
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-7-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819602
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-8-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:817519
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
* Hardware-1-1-9-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:827330
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783194420
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783194420-65c8bf6b44-5bmdx: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:--------------------|:----------------|:------------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 72 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 3313.16 | 0.00 | 50.59 | 0.00 | 175.11 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 71 | randread | 4k | 80 | libaio | 0 | 0 | 50 | 1 | 4361.20 | 0.00 | 49.02 | 0.00 | 206.57 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 72 | randread | 4k | 96 | libaio | 0 | 0 | 50 | 1 | 4002.56 | 0.00 | 65.27 | 0.00 | 396.36 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 72 | randread | 4k | 112 | libaio | 0 | 0 | 50 | 1 | 4200.43 | 0.00 | 79.17 | 0.00 | 413.14 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 70 | randread | 4k | 128 | libaio | 0 | 0 | 50 | 1 | 5286.49 | 0.00 | 78.12 | 0.00 | 438.30 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 65 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1822.94 | 0.00 | 141.56 | 0.00 | 250.61 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7-1-1 | Hardware-1-1-7 | Hardware-1-1-7-1 | 1 | 7 | 1 | 1 | 64 | randwrite | 4k | 80 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2241.12 | 0.00 | 152.04 | 0.00 | 250.61 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8-1-1 | Hardware-1-1-8 | Hardware-1-1-8-1 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 96 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2578.38 | 0.00 | 152.04 | 0.00 | 263.19 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9-1-1 | Hardware-1-1-9 | Hardware-1-1-9-1 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 112 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2552.31 | 0.00 | 173.02 | 0.00 | 341.84 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10-1-1 | Hardware-1-1-10 | Hardware-1-1-10-1 | 1 | 10 | 1 | 1 | 64 | randwrite | 4k | 128 | libaio | 0 | 0 | 50 | 1 | 0.00 | 3076.56 | 0.00 | 168.82 | 0.00 | 291.50 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:----------------|:----------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 72 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 3313.16 | 0.00 | 50.59 | 0.00 | 175.11 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 71 | randread | 4k | 80 | libaio | 0 | 0 | 50 | 4361.20 | 0.00 | 49.02 | 0.00 | 206.57 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 72 | randread | 4k | 96 | libaio | 0 | 0 | 50 | 4002.56 | 0.00 | 65.27 | 0.00 | 396.36 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 72 | randread | 4k | 112 | libaio | 0 | 0 | 50 | 4200.43 | 0.00 | 79.17 | 0.00 | 413.14 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 70 | randread | 4k | 128 | libaio | 0 | 0 | 50 | 5286.49 | 0.00 | 78.12 | 0.00 | 438.30 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 65 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1822.94 | 0.00 | 141.56 | 0.00 | 250.61 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7 | Hardware-1-1-7 | 1 | 7 | 1 | 1 | 64 | randwrite | 4k | 80 | libaio | 0 | 0 | 50 | 0.00 | 2241.12 | 0.00 | 152.04 | 0.00 | 250.61 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8 | Hardware-1-1-8 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 96 | libaio | 0 | 0 | 50 | 0.00 | 2578.38 | 0.00 | 152.04 | 0.00 | 263.19 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9 | Hardware-1-1-9 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 112 | libaio | 0 | 0 | 50 | 0.00 | 2552.31 | 0.00 | 173.02 | 0.00 | 341.84 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10 | Hardware-1-1-10 | 1 | 10 | 1 | 1 | 64 | randwrite | 4k | 128 | libaio | 0 | 0 | 50 | 0.00 | 3076.56 | 0.00 | 168.82 | 0.00 | 291.50 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 67.31 | 2.51 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 75.69 | 4.20 | 0.02 | 0.02 |
| Hardware-1-1-3-1 | 75.00 | 3.93 | 0.03 | 2.17 |
| Hardware-1-1-4-1 | 64.23 | 1.85 | 0.02 | 0.02 |
| Hardware-1-1-5-1 | 72.15 | 2.09 | 0.03 | 4.03 |
| Hardware-1-1-6-1 | 66.85 | 3.03 | 0.02 | 0.02 |
| Hardware-1-1-7-1 | 62.88 | 2.02 | 0.02 | 0.02 |
| Hardware-1-1-8-1 | 73.73 | 2.75 | 0.02 | 0.02 |
| Hardware-1-1-9-1 | 69.06 | 2.72 | 0.02 | 0.02 |
| Hardware-1-1-10-1 | 68.84 | 1.95 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.50 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.54 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.52 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.54 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.52 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.53 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-7-1 | 0.47 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-8-1 | 0.44 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-9-1 | 0.53 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-10-1 | 0.51 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
-rnn pins the SUT pod to a specific node; without it, the scheduler may place different commands
on different nodes, so results are only directly comparable across commands if -rnn is set
consistently.
3. Numjobs sweep at fixed queue depth (elbow check)
Fixes -xfid 64 (the elbow found above) and sweeps -nbt (numjobs per pod) instead of depth: if
IOPS keep climbing with more threads at the same depth, 64 was a per-queue submission limit, not
a real device ceiling; if IOPS stay flat, 64 is the actual hardware limit.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randread,randwrite \
-xfbs 4k \
-xfid 64 \
-xfe libaio \
-nbp 1 \
-nbt 1,2,4,8,16 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 80Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_numjobs_sweep.log
Result
docs_hardware_fio_numjobs_sweep.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 1662s
* Code: 1783190728
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randread', 'randwrite'].
* Block size(s) swept: ['4k'].
* Queue depth(s) swept: [64].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 80Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1, 2, 4, 8, 16] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823734
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-10-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:815983
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826418
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821858
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:828969
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821433
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818500
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-7-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:827363
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-8-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818057
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
* Hardware-1-1-9-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821903
* volume_size:80.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783190728
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783190728-748c5bf8b6-6grqm: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:--------------------|:----------------|:------------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 4338.31 | 0.00 | 54.26 | 0.00 | 152.04 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 79 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 2 | 4703.97 | 0.00 | 89.65 | 0.00 | 497.03 | 0.00 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 94 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 4 | 5166.99 | 0.00 | 113.77 | 0.00 | 952.11 | 0.00 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 122 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 8 | 5773.17 | 0.00 | 196.08 | 0.00 | 1803.55 | 0.00 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 187 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 16 | 5295.18 | 0.00 | 1333.79 | 0.00 | 4731.17 | 0.00 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1878.38 | 0.00 | 141.56 | 0.00 | 256.90 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7-1-1 | Hardware-1-1-7 | Hardware-1-1-7-1 | 1 | 7 | 1 | 1 | 64 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 2 | 0.00 | 3213.22 | 0.00 | 168.82 | 0.00 | 265.29 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8-1-1 | Hardware-1-1-8 | Hardware-1-1-8-1 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 4 | 0.00 | 3792.92 | 0.00 | 214.96 | 0.00 | 750.78 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9-1-1 | Hardware-1-1-9 | Hardware-1-1-9-1 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 8 | 0.00 | 4919.47 | 0.00 | 320.86 | 0.00 | 1484.78 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10-1-1 | Hardware-1-1-10 | Hardware-1-1-10-1 | 1 | 10 | 1 | 1 | 64 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 16 | 0.00 | 6830.33 | 0.00 | 488.64 | 0.00 | 2768.24 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:----------------|:----------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 4338.31 | 0.00 | 54.26 | 0.00 | 152.04 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 79 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 4703.97 | 0.00 | 89.65 | 0.00 | 497.03 | 0.00 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 94 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 5166.99 | 0.00 | 113.77 | 0.00 | 952.11 | 0.00 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 122 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 5773.17 | 0.00 | 196.08 | 0.00 | 1803.55 | 0.00 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 187 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 5295.18 | 0.00 | 1333.79 | 0.00 | 4731.17 | 0.00 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1878.38 | 0.00 | 141.56 | 0.00 | 256.90 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7 | Hardware-1-1-7 | 1 | 7 | 1 | 1 | 64 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 3213.22 | 0.00 | 168.82 | 0.00 | 265.29 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8 | Hardware-1-1-8 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 3792.92 | 0.00 | 214.96 | 0.00 | 750.78 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9 | Hardware-1-1-9 | 1 | 9 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 4919.47 | 0.00 | 320.86 | 0.00 | 1484.78 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10 | Hardware-1-1-10 | 1 | 10 | 1 | 1 | 64 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 6830.33 | 0.00 | 488.64 | 0.00 | 2768.24 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 70.17 | 2.29 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 70.96 | 1.99 | 0.03 | 4.03 |
| Hardware-1-1-3-1 | 81.90 | 3.40 | 0.03 | 4.03 |
| Hardware-1-1-4-1 | 91.72 | 1.83 | 0.03 | 4.03 |
| Hardware-1-1-5-1 | 108.61 | 1.85 | 0.03 | 4.03 |
| Hardware-1-1-6-1 | 68.42 | 2.56 | 0.02 | 0.02 |
| Hardware-1-1-7-1 | 73.46 | 2.33 | 0.02 | 0.02 |
| Hardware-1-1-8-1 | 71.69 | 4.00 | 0.03 | 0.03 |
| Hardware-1-1-9-1 | 72.63 | 1.74 | 0.03 | 0.03 |
| Hardware-1-1-10-1 | 61.99 | 1.61 | 0.03 | 0.04 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.54 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.57 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.54 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.57 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.58 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.44 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-7-1 | 0.58 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-8-1 | 0.55 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-9-1 | 0.55 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-10-1 | 0.50 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per -nbt (numjobs) value, at the fixed queue depth set by
-xfid.
4. Block-size sweep at fixed queue depth (throughput curve)
Also fixes -xfid 64, but sweeps -xfbs instead of numjobs: finds the best block size at the
queue depth already identified as the elbow, and shows where the workload shifts from IOPS-bound
(small blocks) to bandwidth-bound (large blocks).
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randread,randwrite \
-xfbs 4k,8k,16k,64k,128k,256k,1M \
-xfid 64 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_blocksize_sweep.log
Result
docs_hardware_fio_blocksize_sweep.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 1977s
* Code: 1783192417
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randread', 'randwrite'].
* Block size(s) swept: ['4k', '8k', '16k', '64k', '128k', '256k', '1M'].
* Queue depth(s) swept: [64].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824488
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-10-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821009
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-11-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825466
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-12-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821784
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-13-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819972
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-14-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820658
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821968
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:816479
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818635
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820585
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821033
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-7-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820735
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-8-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825529
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
* Hardware-1-1-9-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821483
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783192417
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783192417-57659bf49b-grq8b: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:--------------------|:----------------|:------------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 4295.42 | 0.00 | 49.02 | 0.00 | 270.53 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 70 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 5709.68 | 0.00 | 37.49 | 0.00 | 94.90 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 72 | randread | 16k | 64 | libaio | 0 | 0 | 50 | 1 | 4000.58 | 0.00 | 46.40 | 0.00 | 259.00 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 71 | randread | 64k | 64 | libaio | 0 | 0 | 50 | 1 | 8313.69 | 0.00 | 30.28 | 0.00 | 113.77 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 69 | randread | 128k | 64 | libaio | 0 | 0 | 50 | 1 | 8716.04 | 0.00 | 25.82 | 0.00 | 130.55 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 69 | randread | 256k | 64 | libaio | 0 | 0 | 50 | 1 | 3598.18 | 0.00 | 50.07 | 0.00 | 488.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7-1-1 | Hardware-1-1-7 | Hardware-1-1-7-1 | 1 | 7 | 1 | 1 | 73 | randread | 1M | 64 | libaio | 0 | 0 | 50 | 1 | 2303.26 | 0.00 | 54.26 | 0.00 | 859.83 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8-1-1 | Hardware-1-1-8 | Hardware-1-1-8-1 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2060.27 | 0.00 | 127.40 | 0.00 | 229.64 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9-1-1 | Hardware-1-1-9 | Hardware-1-1-9-1 | 1 | 9 | 1 | 1 | 63 | randwrite | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2036.00 | 0.00 | 126.35 | 0.00 | 233.83 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10-1-1 | Hardware-1-1-10 | Hardware-1-1-10-1 | 1 | 10 | 1 | 1 | 63 | randwrite | 16k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2054.96 | 0.00 | 114.82 | 0.00 | 214.96 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11-1-1 | Hardware-1-1-11 | Hardware-1-1-11-1 | 1 | 11 | 1 | 1 | 64 | randwrite | 64k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1725.67 | 0.00 | 104.33 | 0.00 | 166.72 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12-1-1 | Hardware-1-1-12 | Hardware-1-1-12-1 | 1 | 12 | 1 | 1 | 63 | randwrite | 128k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1629.86 | 0.00 | 111.67 | 0.00 | 187.70 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13-1-1 | Hardware-1-1-13 | Hardware-1-1-13-1 | 1 | 13 | 1 | 1 | 63 | randwrite | 256k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1406.65 | 0.00 | 127.40 | 0.00 | 191.89 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14-1-1 | Hardware-1-1-14 | Hardware-1-1-14-1 | 1 | 14 | 1 | 1 | 63 | randwrite | 1M | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 715.00 | 0.00 | 240.12 | 0.00 | 851.44 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:----------------|:----------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 71 | randread | 4k | 64 | libaio | 0 | 0 | 50 | 4295.42 | 0.00 | 49.02 | 0.00 | 270.53 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 70 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 5709.68 | 0.00 | 37.49 | 0.00 | 94.90 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 72 | randread | 16k | 64 | libaio | 0 | 0 | 50 | 4000.58 | 0.00 | 46.40 | 0.00 | 259.00 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 71 | randread | 64k | 64 | libaio | 0 | 0 | 50 | 8313.69 | 0.00 | 30.28 | 0.00 | 113.77 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 69 | randread | 128k | 64 | libaio | 0 | 0 | 50 | 8716.04 | 0.00 | 25.82 | 0.00 | 130.55 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 69 | randread | 256k | 64 | libaio | 0 | 0 | 50 | 3598.18 | 0.00 | 50.07 | 0.00 | 488.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7 | Hardware-1-1-7 | 1 | 7 | 1 | 1 | 73 | randread | 1M | 64 | libaio | 0 | 0 | 50 | 2303.26 | 0.00 | 54.26 | 0.00 | 859.83 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8 | Hardware-1-1-8 | 1 | 8 | 1 | 1 | 63 | randwrite | 4k | 64 | libaio | 0 | 0 | 50 | 0.00 | 2060.27 | 0.00 | 127.40 | 0.00 | 229.64 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9 | Hardware-1-1-9 | 1 | 9 | 1 | 1 | 63 | randwrite | 8k | 64 | libaio | 0 | 0 | 50 | 0.00 | 2036.00 | 0.00 | 126.35 | 0.00 | 233.83 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10 | Hardware-1-1-10 | 1 | 10 | 1 | 1 | 63 | randwrite | 16k | 64 | libaio | 0 | 0 | 50 | 0.00 | 2054.96 | 0.00 | 114.82 | 0.00 | 214.96 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11 | Hardware-1-1-11 | 1 | 11 | 1 | 1 | 64 | randwrite | 64k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1725.67 | 0.00 | 104.33 | 0.00 | 166.72 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12 | Hardware-1-1-12 | 1 | 12 | 1 | 1 | 63 | randwrite | 128k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1629.86 | 0.00 | 111.67 | 0.00 | 187.70 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13 | Hardware-1-1-13 | 1 | 13 | 1 | 1 | 63 | randwrite | 256k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1406.65 | 0.00 | 127.40 | 0.00 | 191.89 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14 | Hardware-1-1-14 | 1 | 14 | 1 | 1 | 63 | randwrite | 1M | 64 | libaio | 0 | 0 | 50 | 0.00 | 715.00 | 0.00 | 240.12 | 0.00 | 851.44 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 74.48 | 1.69 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 75.06 | 3.01 | 0.02 | 0.02 |
| Hardware-1-1-3-1 | 69.96 | 2.06 | 0.03 | 4.03 |
| Hardware-1-1-4-1 | 72.50 | 2.23 | 0.03 | 0.03 |
| Hardware-1-1-5-1 | 77.29 | 1.78 | 0.03 | 0.03 |
| Hardware-1-1-6-1 | 71.45 | 2.08 | 0.04 | 0.04 |
| Hardware-1-1-7-1 | 76.74 | 2.35 | 0.09 | 0.09 |
| Hardware-1-1-8-1 | 61.76 | 2.02 | 0.02 | 0.02 |
| Hardware-1-1-9-1 | 71.84 | 2.23 | 0.03 | 0.03 |
| Hardware-1-1-10-1 | 71.85 | 1.66 | 0.02 | 0.02 |
| Hardware-1-1-11-1 | 62.57 | 1.72 | 0.03 | 0.03 |
| Hardware-1-1-12-1 | 69.23 | 2.20 | 0.03 | 0.03 |
| Hardware-1-1-13-1 | 72.29 | 2.25 | 0.04 | 0.04 |
| Hardware-1-1-14-1 | 64.87 | 1.70 | 0.09 | 0.09 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.55 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.44 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.49 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.54 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.49 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.54 | 0.04 | 0.00 | 0.00 |
| Hardware-1-1-7-1 | 0.46 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-8-1 | 0.45 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-9-1 | 0.50 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-10-1 | 0.47 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-11-1 | 0.45 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-12-1 | 0.53 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-13-1 | 0.49 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-14-1 | 0.53 | 0.03 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per block size, at the fixed queue depth set by -xfid.
5. Depth sweep at PostgreSQL’s page size (8k)
The sweeps above use bs=4k as a generic device-IOPS probe. PostgreSQL always issues 8kB pages
(BLCKSZ), so this re-anchors the depth sweep at the actual unit of I/O Postgres uses — the
number that calibrates effective_io_concurrency / maintenance_io_concurrency, PostgreSQL’s
own prefetch-depth knobs.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randread,randwrite \
-xfbs 8k \
-xfid 1,2,4,8,16,32,64,128 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_depth_sweep_8k.log
Result
docs_hardware_fio_depth_sweep_8k.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 2212s
* Code: 1783195883
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randread', 'randwrite'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [1, 2, 4, 8, 16, 32, 64, 128].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822044
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-10-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825002
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-11-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:816640
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-12-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823446
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-13-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824851
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-14-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826533
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-15-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825407
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-16-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820455
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823479
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:829372
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826037
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:829244
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825635
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-7-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823397
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-8-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819244
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
* Hardware-1-1-9-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821267
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783195883
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783195883-97d957696-qdj82: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 15: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 16: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 7: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 8: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 9: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 10: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 11: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 12: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 13: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 14: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 15: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 16: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:--------------------|:----------------|:------------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 70 | randread | 8k | 1 | libaio | 0 | 0 | 50 | 1 | 61.98 | 0.00 | 36.44 | 0.00 | 90.70 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 69 | randread | 8k | 2 | libaio | 0 | 0 | 50 | 1 | 167.54 | 0.00 | 32.11 | 0.00 | 76.02 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 70 | randread | 8k | 4 | libaio | 0 | 0 | 50 | 1 | 403.22 | 0.00 | 26.35 | 0.00 | 66.32 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 72 | randread | 8k | 8 | libaio | 0 | 0 | 50 | 1 | 546.25 | 0.00 | 35.39 | 0.00 | 117.96 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 70 | randread | 8k | 16 | libaio | 0 | 0 | 50 | 1 | 1337.27 | 0.00 | 42.21 | 0.00 | 107.48 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 70 | randread | 8k | 32 | libaio | 0 | 0 | 50 | 1 | 2537.31 | 0.00 | 31.33 | 0.00 | 91.75 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7-1-1 | Hardware-1-1-7 | Hardware-1-1-7-1 | 1 | 7 | 1 | 1 | 71 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 4781.67 | 0.00 | 43.25 | 0.00 | 127.40 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8-1-1 | Hardware-1-1-8 | Hardware-1-1-8-1 | 1 | 8 | 1 | 1 | 71 | randread | 8k | 128 | libaio | 0 | 0 | 50 | 1 | 4743.46 | 0.00 | 66.85 | 0.00 | 734.00 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9-1-1 | Hardware-1-1-9 | Hardware-1-1-9-1 | 1 | 9 | 1 | 1 | 63 | randwrite | 8k | 1 | libaio | 0 | 0 | 50 | 1 | 0.00 | 100.86 | 0.00 | 38.01 | 0.00 | 71.83 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10-1-1 | Hardware-1-1-10 | Hardware-1-1-10-1 | 1 | 10 | 1 | 1 | 63 | randwrite | 8k | 2 | libaio | 0 | 0 | 50 | 1 | 0.00 | 172.16 | 0.00 | 43.78 | 0.00 | 83.36 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11-1-1 | Hardware-1-1-11 | Hardware-1-1-11-1 | 1 | 11 | 1 | 1 | 63 | randwrite | 8k | 4 | libaio | 0 | 0 | 50 | 1 | 0.00 | 279.84 | 0.00 | 50.59 | 0.00 | 112.72 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12-1-1 | Hardware-1-1-12 | Hardware-1-1-12-1 | 1 | 12 | 1 | 1 | 63 | randwrite | 8k | 8 | libaio | 0 | 0 | 50 | 1 | 0.00 | 490.15 | 0.00 | 55.84 | 0.00 | 149.95 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13-1-1 | Hardware-1-1-13 | Hardware-1-1-13-1 | 1 | 13 | 1 | 1 | 65 | randwrite | 8k | 16 | libaio | 0 | 0 | 50 | 1 | 0.00 | 696.68 | 0.00 | 70.78 | 0.00 | 193.99 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14-1-1 | Hardware-1-1-14 | Hardware-1-1-14-1 | 1 | 14 | 1 | 1 | 63 | randwrite | 8k | 32 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1175.87 | 0.00 | 88.60 | 0.00 | 223.35 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-15-1-1 | Hardware-1-1-15 | Hardware-1-1-15-1 | 1 | 15 | 1 | 1 | 64 | randwrite | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 0.00 | 1859.32 | 0.00 | 122.16 | 0.00 | 246.42 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-16-1-1 | Hardware-1-1-16 | Hardware-1-1-16-1 | 1 | 16 | 1 | 1 | 64 | randwrite | 8k | 128 | libaio | 0 | 0 | 50 | 1 | 0.00 | 2449.04 | 0.00 | 162.53 | 0.00 | 434.11 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:----------------|:----------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 70 | randread | 8k | 1 | libaio | 0 | 0 | 50 | 61.98 | 0.00 | 36.44 | 0.00 | 90.70 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 69 | randread | 8k | 2 | libaio | 0 | 0 | 50 | 167.54 | 0.00 | 32.11 | 0.00 | 76.02 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 70 | randread | 8k | 4 | libaio | 0 | 0 | 50 | 403.22 | 0.00 | 26.35 | 0.00 | 66.32 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 72 | randread | 8k | 8 | libaio | 0 | 0 | 50 | 546.25 | 0.00 | 35.39 | 0.00 | 117.96 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 70 | randread | 8k | 16 | libaio | 0 | 0 | 50 | 1337.27 | 0.00 | 42.21 | 0.00 | 107.48 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 70 | randread | 8k | 32 | libaio | 0 | 0 | 50 | 2537.31 | 0.00 | 31.33 | 0.00 | 91.75 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-7 | Hardware-1-1-7 | 1 | 7 | 1 | 1 | 71 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 4781.67 | 0.00 | 43.25 | 0.00 | 127.40 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-8 | Hardware-1-1-8 | 1 | 8 | 1 | 1 | 71 | randread | 8k | 128 | libaio | 0 | 0 | 50 | 4743.46 | 0.00 | 66.85 | 0.00 | 734.00 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-9 | Hardware-1-1-9 | 1 | 9 | 1 | 1 | 63 | randwrite | 8k | 1 | libaio | 0 | 0 | 50 | 0.00 | 100.86 | 0.00 | 38.01 | 0.00 | 71.83 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-10 | Hardware-1-1-10 | 1 | 10 | 1 | 1 | 63 | randwrite | 8k | 2 | libaio | 0 | 0 | 50 | 0.00 | 172.16 | 0.00 | 43.78 | 0.00 | 83.36 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-11 | Hardware-1-1-11 | 1 | 11 | 1 | 1 | 63 | randwrite | 8k | 4 | libaio | 0 | 0 | 50 | 0.00 | 279.84 | 0.00 | 50.59 | 0.00 | 112.72 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-12 | Hardware-1-1-12 | 1 | 12 | 1 | 1 | 63 | randwrite | 8k | 8 | libaio | 0 | 0 | 50 | 0.00 | 490.15 | 0.00 | 55.84 | 0.00 | 149.95 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-13 | Hardware-1-1-13 | 1 | 13 | 1 | 1 | 65 | randwrite | 8k | 16 | libaio | 0 | 0 | 50 | 0.00 | 696.68 | 0.00 | 70.78 | 0.00 | 193.99 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-14 | Hardware-1-1-14 | 1 | 14 | 1 | 1 | 63 | randwrite | 8k | 32 | libaio | 0 | 0 | 50 | 0.00 | 1175.87 | 0.00 | 88.60 | 0.00 | 223.35 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-15 | Hardware-1-1-15 | 1 | 15 | 1 | 1 | 64 | randwrite | 8k | 64 | libaio | 0 | 0 | 50 | 0.00 | 1859.32 | 0.00 | 122.16 | 0.00 | 246.42 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-16 | Hardware-1-1-16 | 1 | 16 | 1 | 1 | 64 | randwrite | 8k | 128 | libaio | 0 | 0 | 50 | 0.00 | 2449.04 | 0.00 | 162.53 | 0.00 | 434.11 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 73.15 | 2.30 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 65.66 | 2.04 | 0.03 | 4.03 |
| Hardware-1-1-3-1 | 73.03 | 2.69 | 0.02 | 0.02 |
| Hardware-1-1-4-1 | 65.29 | 1.79 | 0.02 | 0.02 |
| Hardware-1-1-5-1 | 71.24 | 1.98 | 0.02 | 0.02 |
| Hardware-1-1-6-1 | 65.16 | 2.14 | 0.02 | 0.02 |
| Hardware-1-1-7-1 | 72.52 | 4.75 | 0.02 | 0.02 |
| Hardware-1-1-8-1 | 75.68 | 2.36 | 0.03 | 2.44 |
| Hardware-1-1-9-1 | 65.55 | 2.49 | 0.02 | 0.02 |
| Hardware-1-1-10-1 | 68.96 | 2.28 | 0.08 | 0.08 |
| Hardware-1-1-11-1 | 66.96 | 1.66 | 0.02 | 0.02 |
| Hardware-1-1-12-1 | 58.58 | 1.69 | 0.02 | 0.02 |
| Hardware-1-1-13-1 | 65.96 | 1.79 | 0.02 | 0.02 |
| Hardware-1-1-14-1 | 64.07 | 2.03 | 0.03 | 0.03 |
| Hardware-1-1-15-1 | 67.91 | 1.47 | 0.02 | 0.02 |
| Hardware-1-1-16-1 | 64.59 | 1.68 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:------------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.49 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.50 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.52 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.54 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.52 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.48 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-7-1 | 0.55 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-8-1 | 0.43 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-9-1 | 0.58 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-10-1 | 0.51 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-11-1 | 0.54 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-12-1 | 0.44 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-13-1 | 0.50 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-14-1 | 0.52 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-15-1 | 0.50 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-16-1 | 0.57 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per queue depth, at the 8k block size set by -xfbs.
6. random_page_cost calibration
Sequential vs. random read at the same block size and depth. The latency/throughput ratio
between the two rounds gives a device-specific number to replace random_page_cost’s
spinning-disk-era default of 4.0 (relative to seq_page_cost=1.0) — often closer to 1.1-1.5 on
NVMe.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw read,randread \
-xfbs 8k \
-xfid 64 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_random_page_cost.log
Result
docs_hardware_fio_random_page_cost.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 316s
* Code: 1783198124
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['read', 'randread'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [64].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822444
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198124
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825522
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198124
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783198124-5d959fb658-s85rs: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 71 | read | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 4745.50 | 0.00 | 26.08 | 0.00 | 86.51 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 71 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 1 | 4090.29 | 0.00 | 45.88 | 0.00 | 229.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 71 | read | 8k | 64 | libaio | 0 | 0 | 50 | 4745.50 | 0.00 | 26.08 | 0.00 | 86.51 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 71 | randread | 8k | 64 | libaio | 0 | 0 | 50 | 4090.29 | 0.00 | 45.88 | 0.00 | 229.64 | 0.00 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 70.73 | 1.84 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 70.86 | 5.29 | 0.03 | 4.03 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.44 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.49 | 0.02 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table has one row for read (sequential) and one for randread, both at the same
block size and queue depth.
7. WAL sync-write latency (fsync)
Sequential 8k write + fsync after every write, depth 1, single thread — “how fast can one backend
commit with synchronous_commit=on and no batching” (max TPS ≈ 1/latency).
wal_sync_method=fsync.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw write \
-xfbs 8k \
-xfid 1 \
-xfe libaio \
-xfsy 1 \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_wal_sync_fsync.log
Result
docs_hardware_fio_wal_sync_fsync.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 214s
* Code: 1783198464
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['write'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [1].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [1].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820118
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198464
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783198464-779ccffd68-ldt2x: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 62 | write | 8k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 48.92 | 0.00 | 29.75 | 0.00 | 467.66 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 62 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 48.92 | 0.00 | 29.75 | 0.00 | 467.66 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 66.46 | 1.71 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.44 | 0.00 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
hardware_fio_write_iops and the write latency percentiles in the Per Phase table are the
relevant columns for this single-row result.
8. WAL sync-write latency (fdatasync)
Same as above but fdatasync instead of fsync. fdatasync skips the inode-metadata sync fsync
does, and is PostgreSQL’s Linux default (wal_sync_method=fdatasync) — compare its latency
against the fsync run above to confirm it is actually cheaper on this storage.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw write \
-xfbs 8k \
-xfid 1 \
-xfe libaio \
-xffd 1 \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_wal_sync_fdatasync.log
Result
docs_hardware_fio_wal_sync_fdatasync.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 182s
* Code: 1783198702
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['write'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [1].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [1].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825046
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198702
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783198702-85dd59cf54-f67jc: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 8k | 1 | libaio | 0 | 1 | 50 | 1 | 0.00 | 177.61 | 0.00 | 25.03 | 0.00 | 58.46 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 8k | 1 | libaio | 0 | 1 | 50 | 0.00 | 177.61 | 0.00 | 25.03 | 0.00 | 58.46 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 64.00 | 2.58 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.50 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
hardware_fio_write_iops and the write latency percentiles are the relevant columns to compare
against command 7’s fsync result.
9. WAL group-commit scaling
Same sync-write profile, sweeping concurrent committing backends (-nbt) instead of a single
one. If aggregate fsyncs/sec keeps climbing with more concurrent writers, the storage/controller
coalesces concurrent commits well; if it flattens immediately, tune commit_delay /
commit_siblings in Postgres to force batching in software instead.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw write \
-xfbs 8k \
-xfid 1 \
-xfe libaio \
-xfsy 1 \
-nbp 1 \
-nbt 1,2,4,8,16,32 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 150Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_wal_group_commit.log
Result
docs_hardware_fio_wal_group_commit.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 856s
* Code: 1783198907
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['write'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [1].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [1].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 150Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1, 2, 4, 8, 16, 32] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826576
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824462
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824318
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818035
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823012
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824871
* volume_size:150.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783198907
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783198907-565b848d8d-p7zpw: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 64 | write | 8k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 64.94 | 0.00 | 17.43 | 0.00 | 429.92 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 2 | 0.00 | 181.98 | 0.00 | 57.93 | 0.00 | 100.14 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 4 | 0.00 | 476.06 | 0.00 | 50.07 | 0.00 | 74.97 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 8 | 0.00 | 852.53 | 0.00 | 50.59 | 0.00 | 104.33 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 16 | 0.00 | 1396.17 | 0.00 | 50.07 | 0.00 | 91.75 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 32 | 0.00 | 2593.54 | 0.00 | 58.46 | 0.00 | 115.87 | 32 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 64 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 64.94 | 0.00 | 17.43 | 0.00 | 429.92 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 181.98 | 0.00 | 57.93 | 0.00 | 100.14 | 2 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 476.06 | 0.00 | 50.07 | 0.00 | 74.97 | 4 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 852.53 | 0.00 | 50.59 | 0.00 | 104.33 | 8 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 1396.17 | 0.00 | 50.07 | 0.00 | 91.75 | 16 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 2593.54 | 0.00 | 58.46 | 0.00 | 115.87 | 32 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 64.08 | 2.65 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 63.25 | 1.56 | 0.02 | 0.02 |
| Hardware-1-1-3-1 | 60.88 | 1.96 | 0.02 | 0.02 |
| Hardware-1-1-4-1 | 69.21 | 1.75 | 0.02 | 0.02 |
| Hardware-1-1-5-1 | 69.16 | 2.91 | 0.03 | 0.03 |
| Hardware-1-1-6-1 | 71.18 | 3.49 | 0.04 | 0.04 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.56 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.57 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.49 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.57 | 0.04 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.62 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.52 | 0.04 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per concurrent-writer count set by -nbt.
10. WAL record-size sweep
Same sync-write profile, sweeping the WAL record size instead of backend count. Bigger
transactions (or post-checkpoint full_page_writes bursts) write more before fsync — this shows
how sync-write latency grows with record size.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw write \
-xfbs 1k,8k,16k,32k,64k \
-xfid 1 \
-xfe libaio \
-xfsy 1 \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_wal_record_size.log
Result
docs_hardware_fio_wal_record_size.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 723s
* Code: 1783199789
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['write'].
* Block size(s) swept: ['1k', '8k', '16k', '32k', '64k'].
* Queue depth(s) swept: [1].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [1].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823240
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783199789
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:831169
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783199789
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824028
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783199789
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:827995
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783199789
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:819882
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783199789
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783199789-d99d9b8c8-dmq44: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 1k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 84.38 | 0.00 | 52.17 | 0.00 | 79.17 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 145.23 | 0.00 | 33.42 | 0.00 | 63.70 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 63 | write | 16k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 94.12 | 0.00 | 33.42 | 0.00 | 63.18 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 63 | write | 32k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 127.30 | 0.00 | 35.39 | 0.00 | 74.97 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 62 | write | 64k | 1 | libaio | 1 | 0 | 50 | 1 | 0.00 | 53.67 | 0.00 | 65.80 | 0.00 | 88.60 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 1k | 1 | libaio | 1 | 0 | 50 | 0.00 | 84.38 | 0.00 | 52.17 | 0.00 | 79.17 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 63 | write | 8k | 1 | libaio | 1 | 0 | 50 | 0.00 | 145.23 | 0.00 | 33.42 | 0.00 | 63.70 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 63 | write | 16k | 1 | libaio | 1 | 0 | 50 | 0.00 | 94.12 | 0.00 | 33.42 | 0.00 | 63.18 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 63 | write | 32k | 1 | libaio | 1 | 0 | 50 | 0.00 | 127.30 | 0.00 | 35.39 | 0.00 | 74.97 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 62 | write | 64k | 1 | libaio | 1 | 0 | 50 | 0.00 | 53.67 | 0.00 | 65.80 | 0.00 | 88.60 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 69.84 | 2.70 | 0.02 | 0.02 |
| Hardware-1-1-2-1 | 63.40 | 1.72 | 0.02 | 0.02 |
| Hardware-1-1-3-1 | 62.02 | 1.64 | 0.02 | 0.02 |
| Hardware-1-1-4-1 | 69.61 | 1.90 | 0.02 | 0.02 |
| Hardware-1-1-5-1 | 61.78 | 1.75 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.45 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.55 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.52 | 0.04 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.44 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.52 | 0.02 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per block size set by -xfbs, at fixed queue depth 1.
11. Checkpoint writeback bandwidth
Large-block sequential writes without a per-write fsync, approximating how fast checkpointer/bgwriter can flush dirty pages during a checkpoint.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw write \
-xfbs 1M,4M,16M \
-xfid 4,16 \
-xfe libaio \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_checkpoint_writeback.log
Result
docs_hardware_fio_checkpoint_writeback.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 864s
* Code: 1783200536
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['write'].
* Block size(s) swept: ['1M', '4M', '16M'].
* Queue depth(s) swept: [4, 16].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [0].
* Fdatasync interval(s) swept: [0].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:831367
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820536
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:827659
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821051
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
* Hardware-1-1-5-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:825418
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
* Hardware-1-1-6-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821889
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783200536
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783200536-f46bc844-kqq79: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 5: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 6: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 1M | 4 | libaio | 0 | 0 | 50 | 1 | 0.00 | 76.80 | 0.00 | 104.33 | 0.00 | 164.63 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 63 | write | 1M | 16 | libaio | 0 | 0 | 50 | 1 | 0.00 | 237.50 | 0.00 | 143.65 | 0.00 | 212.86 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 63 | write | 4M | 4 | libaio | 0 | 0 | 50 | 1 | 0.00 | 49.68 | 0.00 | 139.46 | 0.00 | 206.57 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 63 | write | 4M | 16 | libaio | 0 | 0 | 50 | 1 | 0.00 | 142.58 | 0.00 | 235.93 | 0.00 | 362.81 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5-1-1 | Hardware-1-1-5 | Hardware-1-1-5-1 | 1 | 5 | 1 | 1 | 63 | write | 16M | 4 | libaio | 0 | 0 | 50 | 1 | 0.00 | 3.11 | 0.00 | 1182.79 | 0.00 | 1283.46 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6-1-1 | Hardware-1-1-6 | Hardware-1-1-6-1 | 1 | 6 | 1 | 1 | 63 | write | 16M | 16 | libaio | 0 | 0 | 50 | 1 | 0.00 | 3.08 | 0.00 | 5536.48 | 0.00 | 5737.81 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 63 | write | 1M | 4 | libaio | 0 | 0 | 50 | 0.00 | 76.80 | 0.00 | 104.33 | 0.00 | 164.63 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 63 | write | 1M | 16 | libaio | 0 | 0 | 50 | 0.00 | 237.50 | 0.00 | 143.65 | 0.00 | 212.86 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 63 | write | 4M | 4 | libaio | 0 | 0 | 50 | 0.00 | 49.68 | 0.00 | 139.46 | 0.00 | 206.57 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 63 | write | 4M | 16 | libaio | 0 | 0 | 50 | 0.00 | 142.58 | 0.00 | 235.93 | 0.00 | 362.81 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-5 | Hardware-1-1-5 | 1 | 5 | 1 | 1 | 63 | write | 16M | 4 | libaio | 0 | 0 | 50 | 0.00 | 3.11 | 0.00 | 1182.79 | 0.00 | 1283.46 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
| Hardware-1-1-6 | Hardware-1-1-6 | 1 | 6 | 1 | 1 | 63 | write | 16M | 16 | libaio | 0 | 0 | 50 | 0.00 | 3.08 | 0.00 | 5536.48 | 0.00 | 5737.81 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 62.71 | 2.49 | 0.03 | 0.03 |
| Hardware-1-1-2-1 | 67.90 | 1.91 | 0.04 | 0.04 |
| Hardware-1-1-3-1 | 59.10 | 1.61 | 0.04 | 0.04 |
| Hardware-1-1-4-1 | 72.89 | 2.15 | 0.09 | 0.09 |
| Hardware-1-1-5-1 | 64.14 | 1.67 | 0.09 | 0.09 |
| Hardware-1-1-6-1 | 62.89 | 2.54 | 0.27 | 0.27 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.57 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.57 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.57 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.44 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-5-1 | 0.56 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-6-1 | 0.53 | 0.02 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
The Per Phase table lists one row per block size/queue depth combination; bandwidth can be
derived from hardware_fio_write_iops × hardware_fio_bs.
12. OLTP/WAL contention proxy
A single-profile approximation of foreground OLTP traffic contending with WAL flushes on one
queue: mixed random read/write with fsync on the write side. This is not the same as true
concurrent checkpoint+WAL+OLTP contention — that needs several parallel benchmarker jobs with
different fio profiles running in the same round, which bexhoma does not yet support for Hardware
(the closest existing precedent is the TPC-H refresh-stream mechanism, see
bexhoma/experiments/CLAUDE.md §8, which could be adapted for this in the future). This proxy is
what is achievable today with the single-profile-per-round model used throughout this page.
bexhoma hardware \
-dbms Hardware \
-xht fio \
-xts 4G \
-xtd 60 \
-xfrw randrw \
-xfmx 70 \
-xfbs 8k \
-xfid 64 \
-xfe libaio \
-xfsy 1 \
-nbp 1 \
-nbt 1 \
-ne 1 \
-m \
-ms $BEXHOMA_MS \
-tr \
-rsr \
-rss 50Gi \
-rst $BEXHOMA_STORAGE_CLASS \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_fio_oltp_wal_contention_proxy.log
Result
docs_hardware_fio_oltp_wal_contention_proxy.log
## Show Summary
### Workload
Hardware Benchmark (fio)
* Type: hardware
* Duration: 183s
* Code: 1783201425
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: fio.
* Test file size is '4G', duration per round is 60s.
* I/O pattern(s) swept: ['randrw'].
* Block size(s) swept: ['8k'].
* Queue depth(s) swept: [64].
* I/O engine(s) swept: ['libaio'].
* Fsync interval(s) swept: [1].
* Fdatasync interval(s) swept: [0].
* Read mix percentage(s) swept: [70].
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by sidecar containers.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Database is persisted to disk of type shared and size 50Gi. Persistent storage is removed at experiment start.
* Benchmarking is tested with [1] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:831834
* volume_size:50.0G
* cpu_list:0-223
* requests_cpu:4
* requests_memory:16Gi
* eval_parameters
* code:1783201425
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783201425-655946dbcb-qxgkb: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 72 | randrw | 8k | 64 | libaio | 1 | 0 | 70 | 1 | 1955.51 | 835.77 | 49.02 | 100.14 | 170.92 | 316.67 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 72 | randrw | 8k | 64 | libaio | 1 | 0 | 70 | 1955.51 | 835.77 | 49.02 | 100.14 | 170.92 | 316.67 | 1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 70.17 | 1.80 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.50 | 0.00 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero read or write IOPS
hardware_fio_read_iops and hardware_fio_write_iops in the single result row can be compared
against the isolated randread result from command 5 at the same depth and block size.
Which PostgreSQL parameter each command informs
Command(s) |
Informs |
|---|---|
1, 2 (depth sweep + refinement) |
|
3 (numjobs) |
|
4 (block size) |
Reasoning about checkpoint/bgwriter I/O coalescing |
5, 6 (8k depth sweep, page cost) |
|
7, 8 (WAL sync fsync/fdatasync) |
|
9 (group commit) |
|
10 (WAL record size) |
Expectations around |
11 (checkpoint bandwidth) |
|
12 (OLTP/WAL proxy) |
Sanity check under mixed load |
Sysbench CPU noisy-neighbor benchmarks
Unlike the fio sweeps above, these four commands (-xht sysbench) are not about
characterizing storage — they use no disk I/O at all, so none of them need -rst/-rss/-rsr.
The question they investigate is whether a Kubernetes CPU limit (-lc) on the SUT container
actually isolates co-located workloads from each other, the way it’s supposed to. Command 13
calibrates the per-pod saturation point; 14 and 15 are cheap sanity checks against a single SUT
(no second SUT pod needed); 16 is the actual cross-tenant test.
Each command sets -xtd 60, capping both of run_sysbench.sh’s phases (CPU, then memory) at 60
seconds instead of sysbench’s own short built-in default — without it, a round can finish before
-m/-mc’s Prometheus scrape interval ever samples it, showing no monitoring data at all. Total
round length ends up roughly 1-2 minutes (up to 2x60s, since the memory phase can finish earlier
once its 10G transfers, plus SSH/pod overhead). The result DataFrame’s duration column reports
the actual measured wall-clock length (BEXHOMA_DURATION) for every hardware benchmark — fio or
sysbench — so a suspiciously short round is visible directly in the table rather than only in the
raw log.
References:
sysbench: https://github.com/akopytov/sysbench
Kubernetes CPU limits and CFS quota: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
13. CPU-quota calibration (thread sweep)
A single SUT pod, -lc 2 -rc 2 (request equals limit, so the CPU quota is a hard ceiling, not a
burstable one), sweeping sysbench’s own thread count (-nbt) at a fixed -nbp 1:
bexhoma hardware \
-dbms Hardware \
-xht sysbench \
-xtd 60 \
-nbp 1 \
-nbt 1,2,4,8 \
-ne 1 \
-m \
-mc \
-ms $BEXHOMA_MS \
-tr \
-lc 2 \
-rc 2 \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_sysbench_cpu_quota_calibration.log
The Per Phase table lists one row per thread count, with hardware_sysbench_cpu_events_per_sec
and the -mc CPU usage columns being the ones to compare across rows; the thread count is used as
a fixed parameter in commands 14-16.
Result
docs_hardware_sysbench_cpu_quota_calibration.log
## Show Summary
### Workload
Hardware Benchmark (sysbench)
* Type: hardware
* Duration: 569s
* Code: 1783201632
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: sysbench.
* Duration per round is 60s, capping each of the CPU and memory phases (see images/hardware/benchmarker/run_sysbench.sh).
* Total sysbench thread count(s) swept: [1, 2, 4, 8], split across pod count(s): [1].
* CPU phase: sysbench cpu --cpu-max-prime=20000 (fixed).
* Memory phase: sysbench memory --memory-block-size=1K --memory-total-size=10G (fixed; may finish before the duration cap if this transfers first).
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by a cluster-wide installation.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Benchmarking is tested with [1, 2, 4, 8] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824317
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783201632
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820813
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783201632
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:826667
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783201632
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:821568
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783201632
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783201632-5cbcd87c5f-5rsxq: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 62 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 1216.19 | 60.00 | 0.84 | 6960860.82 | 6797.72 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 62 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2425.42 | 60.00 | 0.86 | 6356430.17 | 6207.45 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 70 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2417.14 | 60.01 | 0.92 | 1147806.39 | 1120.90 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 70 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 8 | 2413.27 | 60.05 | 1.01 | 1072236.03 | 1047.11 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 62 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 1216.19 | 60.00 | 0.84 | 6960860.82 | 6797.72 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 1 | 62 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2425.42 | 60.00 | 0.86 | 6356430.17 | 6207.45 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 1 | 70 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2417.14 | 60.01 | 0.92 | 1147806.39 | 1120.90 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 1 | 70 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 8 | 2413.27 | 60.05 | 1.01 | 1072236.03 | 1047.11 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 49.76 | 1.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 107.62 | 2.00 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 103.64 | 2.00 | 0.01 | 0.01 |
| Hardware-1-1-4-1 | 133.97 | 2.00 | 0.01 | 0.01 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.25 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.30 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 0.31 | 0.02 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 0.25 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero CPU events/sec
14. Harness-overhead sweep (-nbp)
The same fixed total of 4 sysbench threads against the same -lc 2 SUT, but re-partitioned
across a growing number of separate benchmarker pods/SSH sessions (-nbp) instead of more
--threads inside one pod:
bexhoma hardware \
-dbms Hardware \
-xht sysbench \
-xtd 60 \
-nbp 1,2,4 \
-nbt 4 \
-ne 1 \
-m \
-mc \
-ms $BEXHOMA_MS \
-tr \
-lc 2 \
-rc 2 \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_sysbench_nbp_overhead_sweep.log
hardware.py computes threads_per_pod = -nbt / -nbp, so the total sysbench thread count stays
constant across all rounds in the Per Phase table — only the number of separate pods carrying
them (pod_count) changes.
Result
docs_hardware_sysbench_nbp_overhead_sweep.log
## Show Summary
### Workload
Hardware Benchmark (sysbench)
* Type: hardware
* Duration: 450s
* Code: 1783202226
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: sysbench.
* Duration per round is 60s, capping each of the CPU and memory phases (see images/hardware/benchmarker/run_sysbench.sh).
* Total sysbench thread count(s) swept: [4], split across pod count(s): [1, 2, 4].
* CPU phase: sysbench cpu --cpu-max-prime=20000 (fixed).
* Memory phase: sysbench memory --memory-block-size=1K --memory-total-size=10G (fixed; may finish before the duration cap if this transfers first).
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by a cluster-wide installation.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Benchmarking is tested with [4] threads, split into [1, 2, 4] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823668
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202226
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:820400
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202226
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822066
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202226
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783202226-57d4db9fff-2q594: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (2 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (4 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (2 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (4 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 65 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2421.27 | 60.00 | 0.92 | 2159458.19 | 2108.85 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 68 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 1241.33 | 60.00 | 0.83 | 1651070.04 | 1612.37 | 0.00 | 0 |
| Hardware-1-1-2-1-2 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 2 | 65 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 1208.00 | 60.03 | 0.94 | 3275980.07 | 3199.20 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 66 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 609.85 | 60.00 | 0.89 | 3762704.60 | 3674.52 | 0.00 | 0 |
| Hardware-1-1-3-1-2 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 2 | 66 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 612.99 | 60.03 | 0.87 | 3809677.72 | 3720.39 | 0.00 | 0 |
| Hardware-1-1-3-1-3 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 3 | 65 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 611.84 | 60.00 | 0.87 | 3933162.03 | 3840.98 | 0.00 | 0 |
| Hardware-1-1-3-1-4 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 4 | 65 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1 | 607.30 | 60.00 | 0.89 | 3686524.48 | 3600.12 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 65 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2421.27 | 60.00 | 0.92 | 2159458.19 | 2108.85 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 2 | 68 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2449.33 | 60.03 | 0.94 | 4927050.11 | 4811.57 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 4 | 66 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2441.98 | 60.03 | 0.89 | 15192068.83 | 14836.01 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 108.60 | 2.00 | 0.01 | 0.01 |
| Hardware-1-1-2-1 | 115.81 | 2.00 | 0.01 | 0.01 |
| Hardware-1-1-3-1 | 81.57 | 2.00 | 0.02 | 0.02 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.29 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.54 | 0.01 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 1.04 | 0.04 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero CPU events/sec
Result
docs_hardware_sysbench_ne_saturation_sweep.log
## Show Summary
### Workload
Hardware Benchmark (sysbench)
* Type: hardware
* Duration: 680s
* Code: 1783202702
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: sysbench.
* Duration per round is 60s, capping each of the CPU and memory phases (see images/hardware/benchmarker/run_sysbench.sh).
* Total sysbench thread count(s) swept: [2], split across pod count(s): [1].
* CPU phase: sysbench cpu --cpu-max-prime=20000 (fixed).
* Memory phase: sysbench memory --memory-block-size=1K --memory-total-size=10G (fixed; may finish before the duration cap if this transfers first).
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by a cluster-wide installation.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Benchmarking is tested with [2] threads, split into [1] pods.
* Benchmarking is run as [1, 2, 4, 8] times the number of benchmarking pods.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818212
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202702
* Hardware-1-1-2-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:824999
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202702
* Hardware-1-1-3-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:822155
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202702
* Hardware-1-1-4-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:823755
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783202702
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783202702-6d86f5ffdc-lk5nh: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (2 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (4 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (8 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-1 - Experiment 1 Client 2: hardware (2 pods)
* DBMS Hardware-1 - Experiment 1 Client 3: hardware (4 pods)
* DBMS Hardware-1 - Experiment 1 Client 4: hardware (8 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 63 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2433.62 | 60.00 | 0.83 | 6003493.54 | 5862.79 | 0.00 | 0 |
| Hardware-1-1-2-1-1 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 1 | 66 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 1217.40 | 60.01 | 0.89 | 2805994.05 | 2740.23 | 0.00 | 0 |
| Hardware-1-1-2-1-2 | Hardware-1-1-2 | Hardware-1-1-2-1 | 1 | 2 | 1 | 2 | 66 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 1225.65 | 60.00 | 0.89 | 2235382.54 | 2182.99 | 0.00 | 0 |
| Hardware-1-1-3-1-1 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 1 | 71 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 610.10 | 60.00 | 1.01 | 1435383.79 | 1401.74 | 0.00 | 0 |
| Hardware-1-1-3-1-2 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 2 | 75 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 611.81 | 60.07 | 0.97 | 887274.92 | 866.48 | 0.00 | 0 |
| Hardware-1-1-3-1-3 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 3 | 75 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 606.78 | 60.00 | 1.01 | 917705.26 | 896.20 | 0.00 | 0 |
| Hardware-1-1-3-1-4 | Hardware-1-1-3 | Hardware-1-1-3-1 | 1 | 3 | 1 | 4 | 71 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 608.08 | 60.07 | 0.99 | 1112504.59 | 1086.43 | 0.00 | 0 |
| Hardware-1-1-4-1-1 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 1 | 79 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 304.27 | 60.00 | 87.56 | 844755.00 | 824.96 | 0.00 | 0 |
| Hardware-1-1-4-1-2 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 2 | 83 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 298.18 | 60.00 | 89.16 | 612420.38 | 598.07 | 0.00 | 0 |
| Hardware-1-1-4-1-3 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 3 | 78 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 304.57 | 60.04 | 89.16 | 838966.61 | 819.30 | 0.00 | 0 |
| Hardware-1-1-4-1-4 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 4 | 80 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 312.14 | 60.00 | 84.47 | 675932.43 | 660.09 | 0.00 | 0 |
| Hardware-1-1-4-1-5 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 5 | 82 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 302.99 | 60.00 | 87.56 | 578640.28 | 565.08 | 0.00 | 0 |
| Hardware-1-1-4-1-6 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 6 | 82 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 301.18 | 60.00 | 87.56 | 596260.25 | 582.29 | 0.00 | 0 |
| Hardware-1-1-4-1-7 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 7 | 76 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 306.11 | 60.00 | 86.00 | 812929.54 | 793.88 | 0.00 | 0 |
| Hardware-1-1-4-1-8 | Hardware-1-1-4 | Hardware-1-1-4-1 | 1 | 4 | 1 | 8 | 74 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 315.57 | 60.06 | 84.47 | 881652.36 | 860.99 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:---------------|:---------------|-----------------:|---------:|----------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 63 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2433.62 | 60.00 | 0.83 | 6003493.54 | 5862.79 | 0.00 | 0 |
| Hardware-1-1-2 | Hardware-1-1-2 | 1 | 2 | 1 | 2 | 66 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 4 | 2443.05 | 60.01 | 0.89 | 5041376.59 | 4923.22 | 0.00 | 0 |
| Hardware-1-1-3 | Hardware-1-1-3 | 1 | 3 | 1 | 4 | 75 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 8 | 2436.77 | 60.07 | 1.01 | 4352868.56 | 4250.85 | 0.00 | 0 |
| Hardware-1-1-4 | Hardware-1-1-4 | 1 | 4 | 1 | 8 | 83 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 16 | 2445.01 | 60.06 | 89.16 | 5841556.85 | 5704.66 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 112.81 | 2.00 | 0.01 | 0.01 |
| Hardware-1-1-2-1 | 120.76 | 2.00 | 0.01 | 0.01 |
| Hardware-1-1-3-1 | 93.42 | 2.00 | 0.02 | 0.02 |
| Hardware-1-1-4-1 | 128.57 | 2.00 | 0.03 | 0.03 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.26 | 0.00 | 0.00 | 0.00 |
| Hardware-1-1-2-1 | 0.62 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-3-1 | 1.01 | 0.03 | 0.00 | 0.00 |
| Hardware-1-1-4-1 | 1.99 | 0.10 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero CPU events/sec
16. Co-located noisy-neighbor test (-mtn/-mtb container)
The actual cross-tenant test. -mtn 4 -mtb container creates 4 independent SutConfiguration
objects — 4 separate SUT pods, 4 separate cgroups — each -lc 2 -rc 2 like command 13, all
pinned to the same physical node via -rnn:
bexhoma hardware \
-dbms Hardware \
-xht sysbench \
-xtd 60 \
-nbp 1 \
-nbt 2 \
-ne 1 \
-m \
-mc \
-ms $BEXHOMA_MS \
-tr \
-lc 2 \
-rc 2 \
-mtb container \
-mtn 4 \
-rnn $BEXHOMA_NODE_SUT -rnb $BEXHOMA_NODE_BENCHMARK \
run &>$LOG_DIR/docs_hardware_sysbench_noisy_neighbor.log
BEXHOMA_TENANT_BY=container makes every tenant’s benchmarker pod wait on one shared
experiment-level Redis counter (bexhoma-benchmarker-podcount-exp-<experiment>, see
images/hardware/benchmarker/benchmarker.sh) before starting sysbench, so all four 2-thread runs
begin at the same synchronized instant instead of drifting apart with each pod’s own scheduling
jitter — otherwise a pod that happens to start stressing the node a few seconds before another
would make the comparison meaningless. get_summary_benchmark_per_phase_multitenant() groups the
result by (phase, tenant_id), giving one row per co-located SUT pod.
hardware_sysbench_cpu_events_per_sec per tenant can be compared against the single-pod baseline
from command 13 at the same thread count.
Result
docs_hardware_sysbench_noisy_neighbor.log
## Show Summary
### Workload
Hardware Benchmark (sysbench)
* Type: hardware
* Duration: 315s
* Code: 1783203408
* fio/sysbench driver runs the experiment.
* This experiment measures raw hardware I/O (fio) or CPU/memory (sysbench) performance.
* Benchmark tool: sysbench.
* Duration per round is 60s, capping each of the CPU and memory phases (see images/hardware/benchmarker/run_sysbench.sh).
* Total sysbench thread count(s) swept: [2], split across pod count(s): [1].
* CPU phase: sysbench cpu --cpu-max-prime=20000 (fixed).
* Memory phase: sysbench memory --memory-block-size=1K --memory-total-size=10G (fixed; may finish before the duration cap if this transfers first).
* Experiment uses bexhoma version 0.10.2.
* System metrics are monitored by a cluster-wide installation.
* Experiment is limited to DBMS ['Hardware'].
* Benchmarking is fixed to cl-worker19.
* SUT is fixed to cl-worker36.
* Benchmarking is tested with [2] threads, split into [1] pods.
* Benchmarking is run as [1] times the number of benchmarking pods.
* Number of tenants is 4, one container per tenant.
* Experiment is run once.
### Connections
* Hardware-1-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:818012
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783203408
* Hardware-2-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:816638
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783203408
* Hardware-3-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:812371
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783203408
* Hardware-4-1-1-1 uses docker image bexhoma/sut_hardware:0.10.2
* RAM:2164173246464
* CPU:INTEL(R) XEON(R) PLATINUM 8570
* Cores:224
* host:6.8.0-111-generic
* node:cl-worker36
* disk:812991
* cpu_list:0-223
* requests_cpu:2
* requests_memory:16Gi
* limits_cpu:2
* eval_parameters
* code:1783203408
### SUT Container Restarts
* bexhoma-sut-hardware-1-1783203408-76f6fd5d47-ltwzr: 0
* bexhoma-sut-hardware-2-1783203408-56b665cf75-bfphh: 0
* bexhoma-sut-hardware-3-1783203408-544968c7cf-j4lmp: 0
* bexhoma-sut-hardware-4-1783203408-6c947b8479-mph7k: 0
### Workflow
#### Actual
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-2 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-3 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-4 - Experiment 1 Client 1: hardware (1 pods)
#### Planned
* DBMS Hardware-1 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-2 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-3 - Experiment 1 Client 1: hardware (1 pods)
* DBMS Hardware-4 - Experiment 1 Client 1: hardware (1 pods)
### Execution
#### Per Connection
| DBMS | phase | job | experiment_run | client | benchmark_run | child | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_numjobs | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-------------------|:---------------|:-----------------|-----------------:|---------:|----------------:|--------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-----------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-1-1 | Hardware-1-1-1 | Hardware-1-1-1-1 | 1 | 1 | 1 | 1 | 140 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2446.25 | 60.00 | 0.83 | 3554824.71 | 3471.51 | 0.00 | 0 |
| Hardware-2-1-1-1-1 | Hardware-2-1-1 | Hardware-2-1-1-1 | 1 | 1 | 1 | 1 | 113 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2444.48 | 60.00 | 0.83 | 6493159.81 | 6340.98 | 0.00 | 0 |
| Hardware-3-1-1-1-1 | Hardware-3-1-1 | Hardware-3-1-1-1 | 1 | 1 | 1 | 1 | 89 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2447.38 | 60.00 | 0.83 | 3700791.83 | 3614.05 | 0.00 | 0 |
| Hardware-4-1-1-1-1 | Hardware-4-1-1 | Hardware-4-1-1-1 | 1 | 1 | 1 | 1 | 63 | | | 0 | | 0 | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2440.37 | 60.00 | 0.83 | 4050259.50 | 3955.33 | 0.00 | 0 |
#### Per Phase
| DBMS | phase | experiment_run | client | benchmark_run | pod_count | tenant_id | duration | hardware_fio_rw | hardware_fio_bs | hardware_fio_iodepth | hardware_fio_engine | hardware_fio_fsync | hardware_fio_fdatasync | hardware_fio_rwmixread | hardware_fio_read_iops | hardware_fio_write_iops | hardware_fio_read_lat_p95_ms | hardware_fio_write_lat_p95_ms | hardware_fio_read_lat_p99_ms | hardware_fio_write_lat_p99_ms | hardware_threads | hardware_sysbench_cpu_events_per_sec | hardware_sysbench_cpu_total_time_s | hardware_sysbench_cpu_lat_p95_ms | hardware_sysbench_memory_ops_per_sec | hardware_sysbench_memory_throughput_mibps | hardware_sysbench_memory_lat_p95_ms | errors |
|:-----------------|:---------------|-----------------:|---------:|----------------:|------------:|------------:|-----------:|:------------------|:------------------|-----------------------:|:----------------------|---------------------:|-------------------------:|-------------------------:|-------------------------:|--------------------------:|-------------------------------:|--------------------------------:|-------------------------------:|--------------------------------:|-------------------:|---------------------------------------:|-------------------------------------:|-----------------------------------:|---------------------------------------:|--------------------------------------------:|--------------------------------------:|---------:|
| Hardware-1-1-1-0 | Hardware-1-1-1 | 1 | 1 | 1 | 1 | 0 | 140 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2446.25 | 60.00 | 0.83 | 3554824.71 | 3471.51 | 0.00 | 0 |
| Hardware-2-1-1-1 | Hardware-2-1-1 | 1 | 1 | 1 | 1 | 1 | 113 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2444.48 | 60.00 | 0.83 | 6493159.81 | 6340.98 | 0.00 | 0 |
| Hardware-3-1-1-2 | Hardware-3-1-1 | 1 | 1 | 1 | 1 | 2 | 89 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2447.38 | 60.00 | 0.83 | 3700791.83 | 3614.05 | 0.00 | 0 |
| Hardware-4-1-1-3 | Hardware-4-1-1 | 1 | 1 | 1 | 1 | 3 | 63 | | | 0 | | 0 | 0 | 0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 2 | 2440.37 | 60.00 | 0.83 | 4050259.50 | 3955.33 | 0.00 | 0 |
### Monitoring
### Execution phase: SUT deployment
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 90.27 | 2.00 | 0.00 | 0.00 |
| Hardware-2-1-1-1 | 102.15 | 2.00 | 0.01 | 0.01 |
| Hardware-3-1-1-1 | 89.29 | 2.00 | 0.00 | 0.00 |
| Hardware-4-1-1-1 | 97.22 | 2.00 | 0.00 | 0.00 |
### Execution phase: component benchmarker
| DBMS | CPU [CPUs] | Max CPU | Max RAM [Gb] | Max RAM Cached [Gb] |
|:-----------------|-------------:|----------:|---------------:|----------------------:|
| Hardware-1-1-1-1 | 0.22 | 0.01 | 0.00 | 0.00 |
| Hardware-2-1-1-1 | 0.26 | 0.01 | 0.00 | 0.00 |
| Hardware-3-1-1-1 | 0.40 | 0.01 | 0.00 | 0.00 |
| Hardware-4-1-1-1 | 0.26 | 0.01 | 0.00 | 0.00 |
### Tests
* TEST passed: No SUT container restarts
* TEST passed: Execution phase: SUT deployment contains no 0 or NaN in CPU [CPUs]
* TEST passed: Execution phase: component benchmarker contains no 0 or NaN in CPU [CPUs]
* TEST passed: Workflow as planned
* TEST passed: Execution Phase: every round has non-zero CPU events/sec
Adjust Parameters
There are various ways to change parameters.
Manifests
The YAML manifests for the components can be found in
https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/tree/master/k8s
(deploymenttemplate-Hardware.yml, jobtemplate-benchmarking-hardware.yml).
Benchmarker script
The fio invocation itself lives in https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/blob/master/images/hardware/benchmarker/run_fio.sh
Command line
You maybe want to adjust some of the parameters that are set in the file: python hardware.py -h
usage: hardware.py [-h] [-aws] [-db] [-sl] [-ss] [-cx CONTEXT] [-e EXPERIMENT]
[-m] [-ma] [-mc] [-ms MAX_SUT] [-mse MAX_SUT_EXPERIMENT]
[-nc NUM_CONFIG] [-ne NUM_QUERY_EXECUTORS] [-nw NUM_WORKER]
[-nwr NUM_WORKER_REPLICAS] [-nws NUM_WORKER_SHARDS]
[-nlp NUM_LOADING_PODS] [-nlt NUM_LOADING_THREADS]
[-nbp NUM_BENCHMARKING_PODS]
[-nbt NUM_BENCHMARKING_THREADS] [-sf SCALING_FACTOR]
[-t TIMEOUT] [-lr LIMIT_RAM] [-lc LIMIT_CPU]
[-rr REQUEST_RAM] [-rc REQUEST_CPU] [-rct REQUEST_CPU_TYPE]
[-rg REQUEST_GPU] [-rgt REQUEST_GPU_TYPE]
[-rst {None,,local-hdd,shared,ramdisk,cephcsi}]
[-rss REQUEST_STORAGE_SIZE] [-rsr]
[-rnn [REQUEST_NODE_NAME]] [-rnl [REQUEST_NODE_LOADING]]
[-rnb [REQUEST_NODE_BENCHMARKING]] [-mtn MULTI_TENANT_NUM]
[-mtb MULTI_TENANT_BY] [-mtv] [-tr] [--set SETS]
[-dbms [{Hardware} ...]] [-xht {fio,sysbench}]
[-xts HARDWARE_SIZE] [-xtd HARDWARE_DURATION]
[-xfrw FIO_RW] [-xfbs FIO_BS] [-xfid FIO_IODEPTH]
[-xfe FIO_ENGINE] [-xfsy FIO_FSYNC] [-xffd FIO_FDATASYNC]
[-xfmx FIO_RWMIXREAD]
{run,start,summary}
Run Hardware (fio/sysbench) benchmarks against a SUT in Kubernetes. Controls
fio workload shape (read/write pattern, block size, queue depth, engine) or
selects sysbench for CPU/memory benchmarking.
positional arguments:
{run,start,summary} experiment phase: start SUT only, run the benchmark,
or summarize results
options:
-h, --help show this help message and exit
-aws, --aws pin components to AWS EKS node groups
-db, --debug enable debug logging
-sl, --skip-loading skip data loading and start benchmarking immediately
-ss, --skip-shutdown keep SUT pods running after the experiment finishes
-cx CONTEXT, --context CONTEXT
kubectl context to use (default: current context)
-e EXPERIMENT, --experiment EXPERIMENT
resume an existing experiment by its code
-m, --monitoring enable Prometheus monitoring for the SUT
-ma, --monitoring-app
enable application-level metrics collection
-mc, --monitoring-cluster
enable node-level monitoring for the entire cluster
-ms MAX_SUT, --max-sut MAX_SUT
maximum number of DBMS configurations to run in
parallel cluster-wide (default: no limit)
-mse MAX_SUT_EXPERIMENT, --max-sut-experiment MAX_SUT_EXPERIMENT
maximum number of DBMS configurations in this
experiment to run in parallel (default: no limit)
-nc NUM_CONFIG, --num-config NUM_CONFIG
number of experiment repetitions per configuration
-ne NUM_QUERY_EXECUTORS, --num-query-executors NUM_QUERY_EXECUTORS
comma-separated list of parallel client counts to
sweep
-nw NUM_WORKER, --num-worker NUM_WORKER
number of worker nodes for distributed DBMS
-nwr NUM_WORKER_REPLICAS, --num-worker-replicas NUM_WORKER_REPLICAS
number of replicas per worker node
-nws NUM_WORKER_SHARDS, --num-worker-shards NUM_WORKER_SHARDS
number of shards per worker node
-nlp NUM_LOADING_PODS, --num-loading-pods NUM_LOADING_PODS
comma-separated list of total loader pod counts
-nlt NUM_LOADING_THREADS, --num-loading-threads NUM_LOADING_THREADS
comma-separated list of total loader threads (split
across pods)
-nbp NUM_BENCHMARKING_PODS, --num-benchmarking-pods NUM_BENCHMARKING_PODS
comma-separated list of benchmarker pod counts
-nbt NUM_BENCHMARKING_THREADS, --num-benchmarking-threads NUM_BENCHMARKING_THREADS
total benchmarking threads, split evenly across pods
-sf SCALING_FACTOR, --scaling-factor SCALING_FACTOR
scaling factor controlling dataset size
-t TIMEOUT, --timeout TIMEOUT
per-query timeout in seconds
-lr LIMIT_RAM, --limit-ram LIMIT_RAM
RAM limit for the SUT and worker pods (e.g. 64Gi; 0 =
no limit)
-lc LIMIT_CPU, --limit-cpu LIMIT_CPU
CPU limit for the SUT and worker pods (e.g. 4; 0 = no
limit)
-rr REQUEST_RAM, --request-ram REQUEST_RAM
RAM request for the SUT and worker pods (e.g. 16Gi)
-rc REQUEST_CPU, --request-cpu REQUEST_CPU
CPU request for the SUT and worker pods (e.g. 4)
-rct REQUEST_CPU_TYPE, --request-cpu-type REQUEST_CPU_TYPE
require SUT node to carry label cpu=<value>
-rg REQUEST_GPU, --request-gpu REQUEST_GPU
number of GPUs to request for the SUT pod
-rgt REQUEST_GPU_TYPE, --request-gpu-type REQUEST_GPU_TYPE
require SUT node to carry label gpu=<value>
-rst {None,,local-hdd,shared,ramdisk,cephcsi}, --request-storage-type {None,,local-hdd,shared,ramdisk,cephcsi}
storage class for the SUT persistent volume
-rss REQUEST_STORAGE_SIZE, --request-storage-size REQUEST_STORAGE_SIZE
size of the SUT persistent volume (e.g. 10Gi)
-rsr, --request-storage-remove
delete any existing PVC for the SUT before starting
-rnn [REQUEST_NODE_NAME], --request-node-name [REQUEST_NODE_NAME]
pin the SUT pod to this Kubernetes node
-rnl [REQUEST_NODE_LOADING], --request-node-loading [REQUEST_NODE_LOADING]
pin loader pods to this Kubernetes node
-rnb [REQUEST_NODE_BENCHMARKING], --request-node-benchmarking [REQUEST_NODE_BENCHMARKING]
pin benchmarker pods to this Kubernetes node
-mtn MULTI_TENANT_NUM, --multi-tenant-num MULTI_TENANT_NUM
number of tenants for multi-tenant experiments
-mtb MULTI_TENANT_BY, --multi-tenant-by MULTI_TENANT_BY
tenancy granularity: schema, database, or container
-mtv, --multi-tenant-volume
allocate a separate persistent volume per tenant
-tr, --test-result validate that results meet basic correctness
requirements
--set SETS override a deployment parameter, e.g. deployment[sut].
container[dbms].max_worker_processes=128
-dbms [{Hardware} ...], --dbms [{Hardware} ...]
hardware target(s) to test
-xht {fio,sysbench}, --xhardware-type {fio,sysbench}
benchmark tool: fio (disk I/O) or sysbench
(CPU/memory)
-xts HARDWARE_SIZE, --xtest-size HARDWARE_SIZE
fio test file size (e.g. 1G, 64G)
-xtd HARDWARE_DURATION, --xtest-duration HARDWARE_DURATION
fio/sysbench run duration in seconds
-xfrw FIO_RW, --xfio-rw FIO_RW
comma-separated fio I/O patterns to sweep, each in
{write, read, randwrite, randread, randrw}
-xfbs FIO_BS, --xfio-blocksize FIO_BS
comma-separated fio block sizes to sweep (e.g.
4k,64k,1M)
-xfid FIO_IODEPTH, --xfio-iodepth FIO_IODEPTH
comma-separated fio queue depths to sweep
-xfe FIO_ENGINE, --xfio-engine FIO_ENGINE
comma-separated fio ioengines to sweep, each in {sync,
libaio, io_uring}
-xfsy FIO_FSYNC, --xfio-fsync FIO_FSYNC
comma-separated fsync intervals to sweep (0 disables
fsync); use fsync xor fdatasync, not both
-xffd FIO_FDATASYNC, --xfio-fdatasync FIO_FDATASYNC
comma-separated fdatasync intervals to sweep (0
disables fdatasync); use fsync xor fdatasync, not both
-xfmx FIO_RWMIXREAD, --xfio-rwmixread FIO_RWMIXREAD
comma-separated read percentages to sweep when
-xfrw=randrw