Skip to content
This repository has been archived by the owner on Jun 28, 2024. It is now read-only.

metrics: add fio storage metrics test #650

Merged
merged 3 commits into from
Sep 12, 2018
Merged

metrics: add fio storage metrics test #650

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
dbb677e
metrics: storage: fio: Add fio storage metric test
Aug 22, 2018
92abbaa
metrics: report: make grabdata modular
Aug 22, 2018
cbf49e6
metrics: report: storage: add fio test to reportgen
Aug 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions metrics/report/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ Repeat this process if you want to compare multiple sets of results. Note, the
report generation scripts process all subfolders of `tests/metrics/results` when
generating the report.

> **Note:** By default, the `grabdata.sh` script tries to launch some moderately
> large containers (i.e. 8Gbyte RAM) and may fail to produce some results on a memory
> constrained system.

## Report generation

Report generation is provided by the `makereport.sh` script. By default this script
Expand Down
202 changes: 155 additions & 47 deletions metrics/report/grabdata.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,136 @@ SCRIPT_DIR=$(dirname "$(readlink -f "$0")")
source "${SCRIPT_DIR}/../lib/common.bash"
RESULTS_DIR=${SCRIPT_DIR}/../results

# By default we run all the tests
RUN_ALL=1

help() {
usage=$(cat << EOF
Usage: $0 [-h] [options]
Description:
This script gathers a number of metrics for use in the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: odd initial indent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me a while to figure out what you mean..... you mean you'd rather see Description with no indent, and everything below pulled left one tab (and the same for the Options) ? Sure, if that is what you mean, I can do that.
I copied this from another metrics file I think (maybe launch_times.sh). I see other scripts are hard-left aligned (like the ci static checks). I have no particular preference...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's difficult to know how it will display when you run it, but the word "This" in line 1 of your description is indented much further than the other lines "report" (first word of line 2), "configured" (first word of line 3), etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I don't see that on either github, vim or when I run it - but, now you point it out I looked at the raw github source view, and I think I see there is a space/tab disparity - thx!

report generation script. Which tests are run can be
configured on the commandline. Specifically enabling
individual tests will disable the 'all' option, unless
'all' is also specified last.
Options:
-a, Run all tests (default).
-d, Run the density tests.
-h, Print this help.
-n, Run the networking tests.
-s, Run the storage tests.
-t, Run the time tests.
EOF
)
echo "$usage"
}

# Set up the initial state
init() {
metrics_onetime_init

local OPTIND
while getopts "adhnst" opt;do
case ${opt} in
a)
RUN_ALL=1
;;
d)
RUN_DENSITY=1
RUN_ALL=
;;
h)
help
exit 0;
;;
n)
RUN_NETWORK=1
RUN_ALL=
;;
s)
RUN_STORAGE=1
RUN_ALL=
;;
t)
RUN_TIME=1
RUN_ALL=
;;
?)
# parse failure
help
die "Failed to parse arguments"
;;
esac
done
shift $((OPTIND-1))
}

run_density_ksm() {
echo "Running KSM density tests"

# Run the memory footprint test - the main test that
# KSM affects. Run for a sufficient number of containers
# (that gives us a fair view of how memory gets shared across
# containers), and a large enough timeout for KSM to settle.
# If KSM has not settled down by then, just take the measurement.
# 'auto' mode should detect when KSM has settled automatically.
bash density/docker_memory_usage.sh 20 300 auto

# Grab scaling system level footprint data for different sized
# container workloads - with KSM enabled.

# busybox - small container
export PAYLOAD_SLEEP="1"
export PAYLOAD="busybox"
export PAYLOAD_ARGS="tail -f /dev/null"
export PAYLOAD_RUNTIME_ARGS=" -m 2G"
bash density/footprint_data.sh

# mysql - medium sized container
# Need to wait for mysql to boot and settle before we measure
export PAYLOAD_SLEEP="10"
export PAYLOAD="mysql"
PAYLOAD_ARGS=" --innodb_use_native_aio=0 --disable-log-bin"
PAYLOAD_RUNTIME_ARGS=" -m 4G -e MYSQL_ALLOW_EMPTY_PASSWORD=1"
bash density/footprint_data.sh

# elasticsearch - large container
# Need to wait for elasticsearch to boot and settle before we measure
export PAYLOAD_SLEEP="10"
export PAYLOAD="elasticsearch"
PAYLOAD_ARGS=" "
PAYLOAD_RUNTIME_ARGS=" -m 8G"
bash density/footprint_data.sh
}

run_density() {
echo "Running non-KSM density tests"

# Run the density tests - no KSM, so no need to wait for settle
# Set a token short timeout, and use enough containers to get a
# good average measurement.
bash density/docker_memory_usage.sh 20 5
}

run_time() {
echo "Running time tests"
# Run the time tests - take time measures for an ubuntu image, over
# 100 'first and only container' launches.
# NOTE - whichever container you test here must support a full 'date'
# command - busybox based containers (including Alpine) will not work.
bash time/launch_times.sh -i ubuntu -n 100
}

run_storage() {
echo "Running storage tests"

# Enable this if you want to test a volume mount
#export TEST_VOLUME_MOUNT=1
bash storage/fio.sh
}

run_network() {
echo "No network tests to run"
}

# Execute metrics scripts
Expand All @@ -33,55 +160,36 @@ run() {
# rest of the tests, as KSM may introduce some extra noise in the
# results by stealing CPU time for instance.
if [[ -f ${KSM_ENABLE_FILE} ]]; then
save_ksm_settings
trap restore_ksm_settings EXIT QUIT KILL
set_ksm_aggressive

# Run the memory footprint test - the main test that
# KSM affects. Run for 20 containers (that gives us a fair
# view of how memory gets shared across containers), and
# a default timeout of 300s - if KSM has not settled down
# by then, just take the measurement.
# 'auto' mode should detect when KSM has settled automatically.
bash density/docker_memory_usage.sh 20 300 auto

# Grab scaling system level footprint data for different sized
# container workloads - with KSM enabled.

# busybox - small container
export PAYLOAD_SLEEP="1"
export PAYLOAD="busybox"
export PAYLOAD_ARGS="tail -f /dev/null"
export PAYLOAD_RUNTIME_ARGS=" -m 2G"
bash density/footprint_data.sh

# mysql - medium sized container
export PAYLOAD_SLEEP="10"
export PAYLOAD="mysql"
PAYLOAD_ARGS=" --innodb_use_native_aio=0 --disable-log-bin"
PAYLOAD_RUNTIME_ARGS=" -m 4G -e MYSQL_ALLOW_EMPTY_PASSWORD=1"
bash density/footprint_data.sh

# elasticsearch - large container
export PAYLOAD_SLEEP="10"
export PAYLOAD="elasticsearch"
PAYLOAD_ARGS=" "
PAYLOAD_RUNTIME_ARGS=" -m 8G"
bash density/footprint_data.sh

# And now ensure KSM is turned off for the rest of the tests
disable_ksm
# No point enabling and disabling KSM if we have nothing to test.
if [ -n "$RUN_ALL" ] || [ -n "$RUN_DENSITY" ]; then
save_ksm_settings
trap restore_ksm_settings EXIT QUIT KILL
set_ksm_aggressive

run_density_ksm

# And now ensure KSM is turned off for the rest of the tests
disable_ksm
fi
else
echo "No KSM control file, skipping KSM tests"
fi

# Run the time tests - take time measures for an ubuntu image, over
# 100 'first and only container' launches.
# NOTE - whichever container you test here must support a full 'date'
# command - busybox based containers (including Alpine) will not work.
bash time/launch_times.sh -i ubuntu -n 100
if [ -n "$RUN_ALL" ] || [ -n "$RUN_TIME" ]; then
run_time
fi

# Run the density tests - no KSM, so no need to wait for settle
# (so set a token 5s wait). Take the measure across 20 containers.
bash density/docker_memory_usage.sh 20 5
if [ -n "$RUN_ALL" ] || [ -n "$RUN_DENSITY" ]; then
run_density
fi

if [ -n "$RUN_ALL" ] || [ -n "$RUN_STORAGE" ]; then
run_storage
fi

if [ -n "$RUN_ALL" ] || [ -n "$RUN_NETWORK" ]; then
run_network
fi

popd
}
Expand All @@ -92,7 +200,7 @@ finish() {
echo "this script again."
}

init
init "$@"
run
finish

Loading